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EXECUTIVE  SUMMARY 

This  report  describes  in  detail  the  procedures,  methods,  and  resources  Parsons  used  to  complete 
an  Environmental  Security  Technology  Certification  Program  (ESTCP)-funded  demonstration  of 
classification  technologies  for  munitions  response  at  the  Pueblo  Chemical  Depot  (PCD)  Solid 
Waste  Management  Unit  (SWMU)  7  under  ESTCP  project  munitions  response  (MR)-201232. 
The  2015  demonstration  at  PCD  was  conducted  with  three  primary  objectives: 

•  Test  and  validate  detection  and  discrimination  capabilities  of  a  currently  available 
advanced  electromagnetic  induction  sensor  developed  specifically  for  discrimination 
on  real  sites  under  operational  conditions. 

•  Investigate  in  cooperation  with  regulators  and  program  managers  how  classification 
technologies  can  be  implemented  in  munitions  and  explosives  of  concern  (MEC) 
cleanup  operations. 

•  Use  the  results  of  the  demonstration  to  identify  targets  that  will  be  excavated  as  part 
of  a  subsequent  investigation  at  the  site. 

Parsons  had  two  separate  teams  working  on  the  project.  One  team  was  responsible  for  site  setup, 
the  placement  of  40  seed  items  for  use  in  measuring  the  capabilities  of  the  MetalMapper 
advanced  EMI  sensor  tested  during  the  project;  the  second  team  was  responsible  for  the 
collection  of  dynamic  MetalMapper  data  over  approximately  5.4  acres  and  the  cued  survey  of 
1,164  targets  identified  in  the  dynamic  data.  The  MetalMapper  field  collection  effort  took  place 
over  3  weeks  with  average  production  rates  of  0.8  acre  per  day  for  the  dynamic  survey  and  232 
targets/day  for  the  cued  survey. 

The  MetalMapper  is  an  advanced  EMI  system  developed  by  Geometries,  Inc.,  with  support  from 
the  ESTCP.  It  has  three  mutually  orthogonal  transmit  loops  in  the  Z,  Y,  and  X  directions  and 
contains  seven  triaxial  receiver  antennas  inside  the  Z  (bottom)  loop,  allowing  21  independent 
measurements  of  the  transient  secondary  magnetic  field.  Dynamic  data  were  collected  using 
only  the  Z  transmit  loop  and  the  seven  receivers.  Target  locations  for  the  cued  survey  were 
selected  from  the  dynamic  data  set.  An  attempt  was  made  to  select  targets  using  a  dipole  filter 
analysis  method  that  used  data  from  21  receiver  orientations,  but  the  size  of  the  smallest  target  of 
interest,  a  fuze  booster,  and  extremely  high  anomaly  densities  across  the  site  limited  the 
effectiveness  of  this  method.  Dynamic  targets  were  selected  using  only  the  data  from  the 
receiver  Z  (parallel  to  ground  surface)  loops. 

Cued  data  were  collected  using  all  three  transmitters  and  21  receiver  loops.  The  collected  data 
were  inverted  and  analyzed  using  the  UX-Analyze  add-on  to  Geosoft’s  Oasis  montaj  software. 
Once  analysis  was  complete,  a  ranked  dig  list  was  submitted  for  the  site.  For  QC  purposes,  the 
ranked  dig  list  was  compared  to  the  seeds  over  which  cued  data  were  collected. 

Twenty-eight  of  the  1,165  targets  collected  were  unclassified  in  the  ranked  dig  list  because  they 
required  re-shots  that  could  not  be  collected  due  to  time  and  budget  constraints.  Therefore,  the 
effectiveness  of  the  MetalMapper  in  separating  targets  of  interest  (TOI)  from  clutter  in  this  area 
was  determined  using  1,137  targets.  Because  an  intrusive  investigation  was  not  part  of  this 
project,  seeded  test  items  were  the  only  known  TOI  at  the  site.  Ten  seeds  were  covered  by  the 
cued  survey.  Removal  of  the  10  correct  classifications  from  the  “dig”  targets  and  the  10  seeds 
from  the  total  number  of  targets  survey  left  55  targets  classified  as  digs  (TOI  or  inconclusive)  out 


ES-1 


of  a  total  of  1,127  targets.  This  represents  a  95.2%  reduction  in  clutter  digs  even  if  all  55 
remaining  digs  are  false  positives.  The  actual  nature  of  these  targets  will  be  detennined  during 
the  intrusive  investigation  at  the  site,  which  has  not  yet  been  scheduled. 
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1.0  INTRODUCTION 

Up  to  90%  of  excavation  costs  on  most  unexploded  ordnance  (UXO)  /  munitions  and  explosives 
of  concern  (MEC)  projects  are  related  to  removing  scrap  metal  that  does  not  represent  an 
explosive  hazard.  Significant  cost  savings  could  be  achieved  through  the  use  of  geophysical 
discrimination  methods  that  could  reduce  the  number  of  excavations  required  to  remove 
explosive  hazards  from  sites.  The  objective  of  this  project  is  to  demonstrate  the  use  of  advanced 
electromagnetic  induction  (EMI)  sensors  in  dynamic  and  static  data  acquisition  modes  and 
associated  analysis  software.  To  achieve  these  objectives,  a  controlled  test  was  conducted  at  the 
Solid  Waste  Management  Unit  (SWMU)  7  at  the  Pueblo  Chemical  Depot  (PCD). 

This  project  was  perfonned  as  a  demonstration  of  classification  technologies  for  munitions 
response  funded  by  the  Environmental  Security  Technology  Certification  Program  (ESTCP) 
under  Munitions  Response  (MR)  Project  201232.  This  demonstration  was  designed  to  evaluate 
classification  methodology  at  a  site  that  is  known  to  contain  subsurface  MEC  from  open  burn 
operations  performed  between  1953  and  1990,  primarily  for  the  destruction  of  mortars, 
projectiles,  and  rockets.  An  earlier  “mag  and  dig”  clearance  performed  in  1997  removed 
approximately  15,500  pounds  of  munitions  debris  (MD)  and  non-MD  scrap  from  the  site, 
including  18  MEC  items.  This  clearance  was  performed  to  a  depth  of  1  foot  (ft.)  below  ground 
surface,  and  full  quality  assurance/quality  control  (QA/QC)  checks  were  performed  for  only  14 
of  the  72  200-ft.  by  200-ft.  grids  cleared.  The  site  was  selected  for  the  ESTCP  program  for  the 
following  reasons: 

•  The  former  use  of  the  site  as  an  open  burn  area  rather  than  a  firing  range  as  has  been 
typical  on  previous  demonstration  projects 

•  The  unknown  extent  of  MEC  contamination  at  the  site  with  regard  to  the  anomaly  density 
remaining  following  the  1-ft.  clearance  and  with  regard  to  the  success  of  the  earlier 
clearance  because  QA  and/or  QC  checks  were  not  performed  on  many  of  the  grids. 

The  results  of  the  demonstration  will  also  be  available  if/when  an  advanced  classification 
remedial  alternative  is  evaluated  in  a  subsequent  Corrective  Measures  Study  Work  Plan. 

1.1  BACKGROUND 

The  Fiscal  Year  2006  defense  appropriation  contained  funding  for  the  “Development  of 
Advanced,  Sophisticated  Discrimination  Technologies  for  Unexploded  Ordnance  (UXO) 
Cleanup.”  The  ESTCP  responded  by  conducting  a  UXO  discrimination  study  at  the  fonner 
Camp  Sibert,  Alabama.  The  results  of  this  first  demonstration  were  very  encouraging.  The 
conditions  for  discrimination  were  favorable  at  this  site  and  included  a  single  target  of  interest 
(TOI;  4.2-  in.  mortar)  and  benign  topography  and  geology.  All  of  the  classification  approaches 
demonstrated  were  correctly  identified  a  sizable  fraction  of  the  anomalies  as  arising  from 
nonhazardous  items  that  could  be  safely  left  in  the  ground.  Both  commercial  and  advanced 
sensors  produced  very  good  results.  ESTCP  organized  a  number  of  demonstrations  at  MR  sites 
across  the  country  between  2006  and  2015,  generally  with  new  variables  added  to  the 
classification  challenges  at  each  subsequent  site  (e.g.,  increased  target  density,  increased 
response  from  local  geology,  mixed  munition  sizes  ranging  from  small  to  very  large,  wooded 
areas).  Additionally,  the  subsequent  projects  included  the  use  of  smaller,  man-portable  EMI 
sensors  such  as  the  Naval  Research  Laboratory’s  (NRL’s)  TEMTADS  2x2  cart,  Lawrence 
Berkeley  National  Laboratory’s  (LBNL’s)  man-portable  Berkeley  UXO  discriminator  (BUD), 
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and  Black  Tusk  Geophysics’  man-portable  vector  machine  (MPV).  All  of  the  EMI  sensors 
tested  to  date  have  been  quite  successful  in  discriminating  between  TOI  and  clutter. 

The  earlier  demonstration  projects  were  focused  on  proving  that  the  technology  was  effective  by 
comparing  theoretical  dig  lists  to  real-world  sources  by  excavating  all  of  the  targets  at  a  given 
site  and  comparing  the  known  source  results  to  the  predicted  source  results.  More  recent  projects 
have  been  focused  on  leaving  metal  classified  as  non-TOI  in  the  ground  following  the 
completion  of  the  project.  An  ESTCP-  and  U.S.  Army  Corps  of  Engineers-funded  pilot  study  at 
the  fonner  Camp  Beale  in  California  and  a  non-ESTCP-related  removal  action  performed  at  two 
sites  at  the  former  Camp  Sibert  resulted  in  over  7,000  dynamic  target  sources  remaining  un-dug 
at  both  sites,  with  regulator  concurrence.  No  TOI  were  misclassified  at  either  site,  and  prior  to 
the  addition  of  quality  assurance  verification  digs,  the  reduction  in  necessary  clutter  digs  was 
above  90%  for  each. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

This  type  of  approach  has  the  potential  to  reduce  the  number  of  excavations  required  to 
effectively  remove  the  explosive  safety  risk  (MEC)  at  a  given  site,  which  would  result  in 
significant  cost  savings  related  to  the  closure  of  fonnerly  used  defense  sites.  The  cost  savings  are 
expected  to  be  particularly  significant  at  removal  action  sites. 

Past  ESTCP  demonstration  sites  have  focused  on  former  firing  ranges,  while  SWMU  7  was  a 
former  open  burn  ground.  It  was  expected  that  both  the  pattern  of  anomaly  distribution  and  the 
types  of  deformation  found  in  native  MEC  and  MD  items  would  be  different  at  this  site  than  at 
past  sites.  The  general  objective  of  the  project  was  to  determine  if  the  use  of  advanced  EMI 
sensors  (i.e.,  MetalMapper)  would  achieve  the  same  results  at  an  open  burn  ground  as  those 
proven  to  be  achievable  at  most  firing  ranges.  Specific  performance  objectives  were  developed  to 
test  the  MetalMapper.  In  addition  to  the  detection  and  correct  classification  of  TOI,  these  include 
maximizing  the  percentage  of  non-TOI  items  correctly  classified,  identifying  the  types  of  TOI 
prior  to  intrusive  investigation,  identifying  the  general  type  of  non-TOI  (e.g.,  fragment, 
horseshoe,  rebar)  prior  to  intrusive  investigation,  minimizing  the  number  of  targets  classified  as 
“inconclusive”,  and  estimating  target  parameters  such  as  location  and  depth  correctly. 

The  second  objective  of  the  demonstration  was  to  evaluate  the  use  of  classification  technology 
for  MEC  site  characterization  at  sites  similar  to  SWMU  7.  The  classification  results  will  initially 
be  evaluated  to  characterize  the  nature  and  extent  of  contamination  at  the  site  with  limited  or  no 
intrusive  data.  A  separate  project  is  expected  to  include  intrusive  investigation  of  some  of  the 
classified  anomalies. 

1.3  REGULATORY  DRIVERS 

As  part  of  the  cleanup  of  former  Department  of  Defense  (DoD)  sites,  buy-in  is  required  from 
regulatory  agencies  at  the  federal,  state,  and  local  levels.  The  advancement  in  classification 
sensors  and  their  successful  deployment  at  real-world  sites  needs  to  be  documented  for  their  use 
to  be  accepted  by  the  applicable  regulatory  agencies.  Their  acceptance  of  the  use  of  this 
technology  at  sites  for  which  they  are  ultimately  responsible  will  be  particularly  important  with 
the  potential  for  DoD  budget  cuts  to  affect  the  amount  of  funding  that  will  be  available  for  future 
remedial  actions. 
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2.0  TECHNOLOGY 


2.1  TECHNOLOGY  DESCRIPTION 

The  MetalMapper  is  an  advanced  EMI  system  developed  by  Geometries,  Inc.,  with  support  from 
the  ESTCP.  The  MetalMapper  draws  elements  of  its  design  from  advanced  systems  currently 
being  developed  by  G&G  Sciences,  Inc.  (supported  by  Naval  Sea  Systems  Command,  the 
Strategic  Environmental  Research  and  Development  Program  [SERDP],  and  ESTCP)  and  by 
LBNL  with  support  from  SERDP  and  ESTCP.  It  has  three  mutually  orthogonal  transmit  loops  in 
the  Z,  Y,  and  X  directions  and  contains  seven  triaxial  receiver  antennas  inside  the  Z  (bottom) 
loop.  Typically,  the  transmit  loops  are  driven  with  a  classical  bipolar  pulse-type  time  domain 
electromagnetic  waveform  (i.e.,  alternating  pulse  polarity  with  a  50%  duty-cycle).  Depending  on 
the  survey  mode  (e.g.,  Static/Dynamic),  the  fundamental  frequency  of  transmission  can  be  varied 
over  the  range  1.1  l<f  <810  hertz  (Hz)  as  can  the  transmit  loops  that  are  driven.  The  seven 
receiver  antennas  allow  2 1  independent  measurements  of  the  transient  secondary  magnetic  field 
for  each  transmit  loop  that  is  driven. 

The  data  acquisition  computer  (DAQ)  is  built  around  a  commercially  available  product  from 
National  Instruments.  The  National  Instruments  DAQ  is  a  full-featured  PC  running  Windows  7. 
The  DAQ,  electromagnetic  transmitter,  and  batteries  for  the  system  are  packaged  in  an  aluminum 
case  that  can  be  mounted  on  a  pack  frame,  on  a  separate  cart  such  as  a  hand  truck,  or  on  the 
survey  vehicle  such  as  a  tractor.  The  instrumentation  package  also  includes  two  external 
modules  that  provide  real-time  kinematic  (RTK)  global  positioning  system  location  (GPS)  and 
platform  attitude  (i.e.,  magnetic  heading,  pitch,  and  roll)  data.  These  modules  are  connected  to 
the  DAQ  through  serial  RS232C  ports.  A  block  diagram  of  the  DAQ  system  is  in  Figure  2.1. 


Figure  2.1 

DAQ  and  DAQ  Functional  Block  Diagram 


A)  MM  DAQ  assembly  with  Panasonic  terminal. 


B)  Function  block  diagram  of  MM  system 


The  MetalMapper  has  two  modes  of  data  collection:  dynamic  and  static.  Data  collected  in 
dynamic  mode  results  in  data  files  containing  many  data  samples.  Generally  speaking,  dynamic 
mode  data  are  collected  while  the  antenna  platform  is  in  motion.  Static  mode  data  collection  is 
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employed  for  cued  surveys.  As  its  name  implies,  the  antenna  platform  remains  static  or 
motionless  during  the  period  of  data  acquisition.  Depending  on  the  acquisition  parameters  (e.g., 
sample  period  and  stacking  parameter),  it  can  take  tens  of  seconds  to  complete  a  static 
measurement.  The  results  of  the  static  measurement  are  written  into  a  binary  data  file  containing 
only  a  single  data  point  representing  the  average  (stacked)  result,  usually  over  tens  or  even 
hundreds  of  repetitions  of  the  transmitter’s  base  frequency. 

Data  are  acquired  in  time  blocks  that  consist  of  a  fixed  number  of  transmitter  cycle  “Repeats.” 
Both  the  period  (T)  and  the  repeat  factor  (N)  are  operator  selectable  and  are  varied  in 
multiplicative  factors  of  3.  The  MetalMapper  also  averages  an  operator-specified  number  of 
acquisition  blocks  (NStacks)  together  before  the  acquired  data  are  saved  to  disk.  The  decay 
transients  that  are  received  during  the  off  times  are  stacked  (averaged)  with  appropriate  sign 
changes  for  positive  and  negative  half  cycles.  The  decays  in  an  individual  acquisition  block  are 
stacked,  and  the  decays  in  that  block  are  averaged  with  other  acquisition  blocks  (assuming  the 
operator  has  selected  NStack  greater  than  one).  The  resultant  data  are  saved  as  a  data  point.  A 
photo  of  the  typical  configuration  of  the  instrument  used  for  collecting  both  dynamic  and  cued 
data  is  shown  in  Figure  2.2. 


Figure  2.2 

Antenna  Array  and  Deployment  of  the  MetalMapper  at  PCD 


In  its  present  (third  generation)  form,  the  MetalMapper  has  been  demonstrated  and  scored  at 
numerous  live  site  demonstrations  carried  out  by  ESTCP.  The  performance  of  the  MetalMapper 
at  these  sites  is  documented  in  formal  reports  issued  by  the  various  contractors  working  on  those 
projects. 

2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

There  are  a  few  advanced  EMI  sensors  that  are  similar  to  the  MetalMapper  in  theory,  design  and 
size,  with  the  most  comparable  being  the  2x2  and  the  MPV.  The  TEMTADS  2x2  consists  of  four 
pairs  of  transmit/tri-axial  receive  coils  oriented  in  a  2x2  grid  pattern,  approximately  1  meter  (m) 
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to  a  side.  The  MPV  is  composed  of  a  single  circular  Z-direction  transmitter  with  five  tri-axial 
receivers  mounted  in  a  cross  pattern  within  the  transmitter  housing.  This  configuration  is 
typically  used  for  dynamic  data  collection,  while  two  additional  square  transmitters,  oriented 
orthogonally  to  the  Z-loop,  can  be  added  for  cued  data  collection.  These  instruments  have  been 
part  of  the  ongoing  ESTCP  classification  demonstrations,  and  similar  results  have  been 
documented  for  all  three  during  previous  projects.  The  main  advantage  of  the  MetalMapper  is 
that  it  is  currently  commercially  available,  while  the  other  two  advanced  EMI  sensors  are  fairly 
limited  in  their  availability.  Although  vehicle  transport  is  not  out  of  the  question  for  either  of  the 
other  two  sensors,  a  great  deal  of  thought  and  development  has  gone  into  making  the 
MetalMapper  the  most  vehicle  friendly  of  the  three  sensors.  As  a  result,  production  rates  are 
typically  higher  for  the  MetalMapper,  particularly  for  cued  data  collection. 

While  the  MetalMapper  is  the  most  amenable  to  vehicle-based  operation,  the  greatest  limitation 
of  the  MetalMapper  is  its  size,  both  of  the  sensor  itself  and  of  the  accompanying  computer, 
screen,  and  cables.  The  system  is  designed  primarily  for  vehicle-based  transport  and,  therefore, 
use  in  relatively  flat,  open  fields.  It  is  the  least  effective  of  the  three  sensors  in  wooded  areas. 
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3.0  PERFORMANCE  OBJECTIVES 

The  specific  performance  objectives  for  this  project  are  summarized  in  Table  3.1. 

Table  3.1 


Performance  Objectives  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Dynamic  Data  Collection  Objectives 

Repeatability  of 
instrument 
verification  strip 
(IVS)  measurements 

Amplitude  of  dynamic 
response 

Twice-daily  IVS 
survey  data 

Amplitude  within  25%  of 
standard  response 

Selected  location  within  25 
centimeters  (cm)  of  known 
location 

Spatial  coverage 

Extended  footprint 
coverage 

Mapped  survey 
data 

95%  of  tracks  with  spacing 
no  greater  than  70  cm  and 
none  greater  than  90  cm 

Down  track  point 
spacing 

Point  to  point  spacing 

Mapped  survey 
data 

90%  of  points  with  down- 
track  spacing  <  1 0  cm  and 

98%  <  15  cm1 

Detection  of  all 
targets  of  interest 

Percent  of  seeded 
anomalies  detected 

Locations  of  seed 
items 

Dynamic  target  list 

Dynamic  target  selected 
within  40  cm  of  all  seed  items 

Cued  Data  Collection  Objectives 

Correctly  identify 
seed  items  in  IVS 

Percentage  of  IVS  items 
identified  correctly 

Twice-daily  IVS 
survey  data 

100%  of  the  IVS  items 
identified  correctly  with 
confidence  metric  of  >0.90 

Correctly  position 
MetalMapper 
relative  to  source 

Distance  between 
collection  location  and 
inverted  target  location 

Location  of 
MetalMapper 
during  collection 
Inverted  target 
location 

100%  of  inverted  locations 
within  40  cm  of  collection 
point  unless  re-shot  also 
outside  radius2 

Correctly  position 
MetalMapper 
relative  to  dynamic 
target 

Distance  between 
MetalMapper  collection 
location  and  dynamic 
target  location 

Dynamic  target  list 

Location  of 
MetalMapper 
during  collection 

100%  of  collection  points 
within  40  cm  of  dynamic 
target  location3 
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Table  3.1  (cont) 

Performance  Objectives  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Analysis  Objectives 

Maximize  targets  of 
interest  (TOI) 
retained  on  dig  list 

Percentage  of  TOI  retained. 

Prioritized  anomaly  list 
Blind  seed  list4 

100%  of  the  TOI 
identified  as  dig  targets 

Minimize  non-TOI 
retained  on  dig  list 

Percentage  of  false  alarms 
eliminated. 

Prioritized  anomaly  list 
Blind  seed  list4 

75%  of  non-TOI  left  in 
ground 

Correctly  identify 
type  of  TOI 

Percentage  of  TOI  correctly 
identified  by  group2 

Prioritized  anomaly  list 
Blind  seed  list4 

75%  of  TOI  identified 
correctly 

Correctly  predict 
attributes  of  non- 
TOI 

Percentage  of  non-TOI  for 
which  at  least  one  attribute  is 
correctly  predicted5 

Analyst  justification  for 
non-TOI  classification  for 

50  validation  target 

1 00%  of  predicted  non- 
TOI  qualitatively 
matches  justification 

Correct  estimation  of 
target  location 

Accuracy  of  estimated  target 
parameters  dig  list  targets 
marked  as  “dig”. 

Demonstrator  target 
parameters 

Intrusive  results 

X,  Y  <30  cm  (la) 

Z  <15  cm  ( 1  a) 

1  98%  <15  cm  is  intended  to  account  for  rapid  movement  of  the  GPS  due  to  sled  movement  over  varying  terrain 
over  a  few  points.  Any  length  of  data  with  more  than  1  m  of  point-to-point  spacings  consistently  over  15  cm  will 
be  re-collected  even  if  the  data  set  as  a  whole  passes  this  metric. 

2  In  addition  to  targets  with  both  initial  and  re-shot  inverted  locations  greater  than  40  cm  from  the  collection  point, 
targets  collected  specifically  to  be  within  40  cm  of  a  pick  location  will  not  negatively  affect  this  objective  if 
inverted  locations  are  greater  than  40  cm  from  the  collection  point;  targets  with  dynamic  anomaly  size  of  <  0.20 
m2  will  not  be  re-collected. 

3  Points  with  no  identified  problems  (e.g.,  inversion  offset,  noise)  and  no  other  targets  within  1 .5  m  of  the  target  in 
question  will  not  be  re-collected;  targets  with  dynamic  anomaly  size  of  <  0.20  nr  will  not  be  re-collected. 

4  Typically  the  prioritized  anomaly  list  would  be  compared  to  the  intrusive  results  for  a  demonstration  site.  The 
schedule  for  the  intrusive  investigation  is  unknown,  so  the  seed  items  placed  during  the  project  will  serve  as  the 
basis  of  comparison  for  this  report. 

5  Expected  TOI  groups  will  include  small  (fuze  boosters,  small  ISOs),  medium  (75-mm  projectiles,  60-  and  81-mm 
mortars,  2.36-inch  and  2.75-inch  rockets,  and  medium  ISOs),  medium/large  (3.5-inch  rockets),  and  large 
(projectiles  larger  than  75-mm  and  large  ISOs). 

3.1  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  STRIP  MEASUREMENTS 

The  reliability  of  the  survey  data  depends  on  the  proper  functioning  of  the  survey  equipment. 

This  objective  concerns  the  twice-daily  confirmation  of  sensor  system  performance. 

3.1.1  Metric 

The  metric  for  this  objective  was  the  measured  response  for  each  instrument  verification  strip 

(I VS)  seed  versus  a  standard  value  determined  for  each  over  the  course  of  the  dynamic  survey 

and  the  distance  between  the  selected  target  and  the  known  location  for  each  seed  item. 


3-2 


3.1.2  Data  Requirements 

The  responses  measured  over  each  seed  item  during  twice-daily  IVS  testing  were  compared  to 
the  expected  response,  and  the  X,  Y  location  of  the  response  was  compared  to  the  known 
location  of  each  seed. 

3.1.3  Success  Criteria 

This  objective  was  met  if  the  dynamic  response  for  each  of  the  seed  items  in  the  IVS  strip  was 
within  25%  of  the  expected  value  and  if  the  selected  location  were  within  25  centimeters  (cm)  of 
the  known  location. 

3.2  SPATIAL  COVERAGE 

The  MetalMapper  detection  survey  should  cover  a  maximum  of  the  area  of  interest  so  that  all 
detectable  targets  are  detected.  Targets  are  detectable  if  the  transmitted  field  is  sufficiently  strong 
to  reach  the  target  and  if  the  measured  target  response  is  sufficiently  strong  in  return  to  exceed  a 
given  threshold.  Simulations  suggest  that  there  is  no  loss  of  detectability  when  a  target  is  located 
1 0  cm  to  the  side  of  the  MetalMapper. 

Only  the  five  middle  receivers  (1  to  5  using  the  UX -Analyze  0  to  6  designations  for  the 
receivers)  were  used  in  calculating  response  in  the  dynamic  survey.  The  outermost  of  these 
receivers  are  approximately  25  cm  to  the  outside  of  the  coil.  Adding  10  cm  to  each  side  as 
discussed  above  results  in  an  effective  footprint  of  70  cm. 

3.2.1  Metric 

The  metrics  for  this  objective  were  collected  coverage  area  versus  between-line  gaps  larger  than 
the  footprint  of  the  instrument  (70  cm)  and  versus  between-line  gaps  large  enough  that  there  was 
a  considerable  risk  that  a  TOI  might  be  missed  completely  (90  cm). 

3.2.2  Data  Requirements 

The  percentage  of  the  coverage  area  with  between-line  gaps  larger  than  70  cm  and  90  cm  were 
calculated  for  each  transect  using  the  foot  print  coverage  tool  in  Oasis  montaj’s  UX-Detect  tool. 

3.2.3  Success  Criteria 

This  objective  was  met  if  more  than  95%  of  each  day’s  coverage  area  had  a  line  spacing  of 
70  cm  or  less  and  if  there  were  no  between-line  gaps  of  greater  than  90  cm. 

3.3  DOWN  TRACK  POINT  SPACING 

As  with  the  spatial  coverage,  the  down  track  point  spacing  objective  ensures  that  the  detection 
survey  covers  the  area  of  interest  such  that  no  potential  TOI  are  missed  due  to  the  sensor  moving 
too  quickly  over  the  survey  area.  This  metric  covers  the  distance  between  measurements  along 
the  path  of  the  survey  line. 

3.3.1  Metric 

The  metrics  for  this  objective  were  the  percentage  of  down  line  point-to-point  spacings  greater 
than  10  cm  from  each  other  and  the  percentage  greater  than  15  cm  from  each  other. 
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3.3.2  Data  Requirements 

The  percentage  of  the  coverage  area  with  down  line  gaps  larger  than  10  cm  and  15  cm  were 
calculated  for  each  day’s  data. 

3.3.3  Success  Criteria 

This  objective  was  met  if  more  than  90%  of  each  day’s  point-to-point  spacings  were  less  than 
10  cm  from  each  other  and  if  there  were  no  down-line  gaps  of  greater  than  15  cm. 

3.4  DETECTION  OF  ALL  TARGETS  OF  INTEREST 

To  be  correctly  classified,  all  TOI  must  be  selected  as  targets  in  the  dynamic  survey.  The 
reliability  of  cued  data  depends  on  acceptable  instrument  positioning  during  data  collection  in 
relation  to  the  actual  anomaly  location. 

3.4.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  seeded  anomalies  that  are  within  the  acceptable 
distance  of  the  center  of  the  instrument  during  data  collection  from  the  actual  target  location. 

3.4.2  Data  Requirements 

The  centers  of  all  seed  items  were  measured  using  the  RTK  GPS  when  they  were  placed  in  the 
ground.  Dynamic  target  selections  were  compared  to  the  known  seed  item  locations  as  dynamic 
target  lists  were  submitted,  and  the  horizontal  distance  was  calculated  between  the  seed  locations 
and  the  nearest  dynamic  target  location. 

3.4.3  Success  Criteria 

The  objective  was  considered  met  for  each  seed  item  if  a  dynamic  target  was  within  40  cm  of  the 
measured  seed  location. 

3.5  CORRECTLY  IDENTIFY  SEED  ITEMS  IN  THE  INSTRUMENT  VERIFICATION 
STRIP 

The  IVS  strip  constructed  at  PCD  contained  three  seed  items  and  a  cleared  background  location. 
MetalMapper  data  were  collected  over  the  IVS  twice  daily. 

3.5.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  IVS  items  correctly  classified  during  the 
project. 

3.5.2  Data  Requirements 

Daily  IVS  data  were  collected  and  processed  in  the  same  manner  as  all  other  target  points 
acquired  during  the  project.  Following  analysis,  each  IVS  target  was  labeled  with  an  identified 
source  object  and  a  confidence  metric  that  quantified  the  degree  of  match  between  three 
polarizability  curves  generated  from  the  measured  IVS  data  and  three  polarizability  curves  for  a 
similar  item  in  a  target  library.  For  the  purposes  of  the  IVS,  the  confidence  metric  used  for 
classification  purposes  was  generated  using  [31  :  p2  :  P3  ratios  of  1  :  1  :  1,  where  pi,  P2,  P3  are 
the  polarizabilities  along  principle  axes  of  the  target. 
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3.5.3  Success  Criteria 

The  performance  objective  for  the  project  was  the  correct  classification  of  all  IVS  seed  items 
with  a  confidence  metric  of  0.90  or  higher. 

3.6  CORRECTLY  POSITION  METALMAPPER  RELATIVE  TO  SOURCE 

High-fidelity  estimates  of  a  target’s  principal  polarizability  curves  depend  on  adequate 
illumination  of  the  target  along  each  of  its  principal  axes.  Targets  with  horizontal  offsets  of 
40  cm  or  more  from  the  center  of  the  MetalMapper  are  not  adequately  illuminated,  and  thus  their 
symmetry  properties  are  sometimes  not  apparent.  While  the  goal  was  to  place  each  sensor 
directly  on  top  of  the  source  item,  adjacent  targets  and  geologic  conditions  can  result  in  modeled 
or  fit  locations  for  the  source  object  a  significant  distance  from  the  collection  point. 

3.6.1  Metric 

The  metric  for  this  objective  was  the  distance  between  the  location  of  the  sensor  for  a  given  point 
and  the  modeled  location  for  the  source  object  following  inversion. 

3.6.2  Data  Requirements 

The  MetalMapper  sensor  location  is  determined  during  the  inversion  of  the  collected  data  and  is 
resolved  using  the  location  identified  by  the  GPS  sensor  directly  over  the  middle  of  the  sensor 
and  pitch  and  roll  data  supplied  by  an  inertial  measurement  unit  (IMU).  The  sensor  location  is 
reported  as  the  X  Array  and  Y  Array  channels  in  the  Geosoft  target  database.  The  location  of 
the  source  object  is  also  calculated  during  target  inversion  and  was  defined  as  the  Fit_X[8]  and 
Fit_Y[8]  channels  in  the  Geosoft  target  database.  The  distance  between  these  two  locations  was 
calculated  for  each  cued  data  point. 

3.6.3  Success  Criteria 

The  perfonnance  objective  was  for  all  targets  to  have  modeled  source  locations  within  40  cm  of 
the  center  of  the  sensor  unless  a  re-shot  had  already  been  performed  on  that  target  or  the  point 
was  collected  solely  so  that  a  point  was  collected  within  40  cm  of  a  dynamic  pick  location 
(Section  3.7).  Additionally,  following  the  completion  of  dynamic  data  collection,  it  became 
apparent  that  the  areal  extent  of  the  seed  items  covered  by  the  survey  was  much  larger  than  the 
areal  extent  of  many  of  the  anomalies  detected  at  the  5  times  background  threshold  used  for 
target  selection.  A  0.2-square-meter  (m2)  anomaly  size  threshold  was  applied  to  the  selected 
dynamic  targets  after  cued  data  collection  had  already  started  on  Transect  3  targets.  Offset 
metrics  were  not  applied  to  already-collected  targets  with  areal  extents  less  than  0.2  nr  as  they 
were  considered  too  small  to  be  TOI. 

3.7  CORRECTLY  POSITION  METALMAPPER  RELATIVE  TO  DYNAMIC  TARGET 

The  use  of  the  “dancing  arrows”  display  to  position  the  MetalMapper  during  data  collection  can 
sometimes  lead  to  a  significant  discrepancy  between  the  location  of  the  MetalMapper  during 
collection  and  the  location  of  the  actual  target  being  investigated  as  selected  in  the  dynamic  data. 
The  location  of  the  MetalMapper  relative  to  the  picked  point  was  compared  to  ensure  that  a 
MetalMapper  point  was  collected  within  a  reasonable  distance  of  each  selected  dynamic  target. 
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3.7.1  Metric 

The  metric  for  this  objective  was  the  distance  between  the  location  of  the  MetalMapper  sensor 
for  a  given  point  and  the  selected  dynamic  target  location. 

3.7.2  Data  Requirements 

The  MetalMapper  sensor  location  was  detennined  during  the  inversion  of  the  collected  data  and 
was  resolved  using  the  location  identified  by  the  GPS  sensor  directly  over  the  middle  of  the 
sensor  and  pitch  and  roll  data  supplied  by  the  IMU.  The  sensor  location  was  defined  as  the 
X_Array[8]  and  Y_Array[8]  channels  in  the  Geosoft  target  database.  These  were  compared  to 
the  target  locations  selected  in  the  dynamic  survey  data. 

3.7.3  Success  Criteria 

The  perfonnance  objective  was  for  all  collection  points  to  be  within  40  cm  of  the  intended 
location.  Points  collected  farther  than  40  cm  from  the  dynamic  target  location  were  considered 
acceptable  if  there  were  no  other  dynamic  targets  within  1.5  m  of  the  target  in  question  and  if 
there  were  no  identifiable  problems  with  the  collected  data,  such  as  noisy  data  or  a  poor 
collection  to  fit  offset  (Section  3.6).  As  discussed  in  Section  3.6.3,  offset  metrics  were  not 
applied  to  dynamic  anomalies  with  sizes  less  than  0.2  m  because  they  were  considered  too  small 
to  be  TOI. 

3.8  MAXIMIZE  TARGETS  OF  INTEREST  RETAINED  ON  THE  DIG  LIST 

One  of  the  two  main  objectives  of  this  project  was  to  show  that  classification  could  correctly 
identify  all  seeded  items,  native  UXO,  and  intact  munitions  items  (with  or  without  explosive 
hazard)  remaining  at  the  site  as  targets. 

3.8.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  seed  items  correctly  identified  as  objects  that 
should  be  dug  in  the  final  ranked  dig  list.  The  comparison  of  the  ranked  dig  list  to  all  TOI 
recovered  at  the  site  will  be  performed  after  an  intrusive  investigation  is  performed  at  the  site, 
but  the  current  schedule  for  that  investigation  is  unknown. 

3.8.2  Data  Requirements 

Following  data  collection,  MetalMapper  data  were  analyzed  to  create  a  prioritized  dig  list,  which 
assigned  each  target  to  one  of  three  categories:  1)  TOI  2)  Non-TOI,  or  3)  Inconclusive.  The 
targets  classified  as  either  TOI  or  Inconclusive  were  considered  “dig”  targets.  The  list  of  seed 
items  was  compared  to  those  targets  marked  “dig”  in  the  ranked  dig  list.  Typically,  training 
targets  are  identified  for  excavation  prior  to  finalizing  the  ranked  dig  list;  however,  none  were 
identified  as  necessary  for  this  project.  As  discussed  above,  the  final  list  of  all  TOI  recovered 
during  the  intrusive  investigation  is  necessary  for  a  complete  assessment  of  classification 
performance.  A  comparison  of  the  ranked  dig  list  to  the  final  TOI  list  will  be  performed 
following  the  completion  of  the  intrusive  investigation. 
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3.8.3  Success  Criteria 


The  performance  objective  was  the  correct  identification  of  all  blind  seed  items  as  targets  that 
should  be  intrusively  investigated,  or  “dig”  targets.  The  project  was  considered  successful  if 
100%  of  the  TOI  were  labeled  as  “dig”  targets  in  the  final  ranked  dig  list.  No  distinction  was 
made  between  a  target  correctly  identified  as  TOI  and  a  target  identified  as  Inconclusive  for  this 
objective.  Each  TOI  simply  needed  to  be  indicated  as  a  target  that  should  be  investigated. 
Further  analysis  will  be  performed  for  all  TOI  recovered  during  the  intrusive  investigation 
depending  on  when  it  takes  place. 

3.9  MINIMIZE  NON-TARGETS  OF  INTEREST  ON  THE  DIG  LIST 

This  is  the  second  of  the  two  primary  measures  of  the  effectiveness  of  the  classification 
approach.  In  addition  to  correctly  classifying  TOI,  the  effectiveness  of  the  MetalMapper  in 
discriminating  munitions  is  a  function  of  the  degree  to  which  responses  that  do  not  correspond  to 
TOI  can  be  eliminated  from  consideration  during  the  intrusive  investigation. 

3.9.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  items  classified  as  non-TOI  following  the 
intrusive  investigation  that  were  correctly  identified  as  objects  that  did  not  need  to  be  intrusively 
investigated  in  the  final  ranked  dig  list. 

3.9.2  Data  Requirements 

Following  data  collection,  MetalMapper  data  was  analyzed  to  create  a  prioritized  dig  list,  which 
assigned  either  a  dig  or  no-dig  designation  to  each  target.  The  targets  classified  as  non-TOI  were 
considered  “no  dig”  or  non-TOI  targets.  The  list  of  items  identified  as  non-TOI  on  the  ranked 
dig  list  was  compared  to  the  blind  seed  list.  As  long  as  no  blind  seeds  were  identified  as  non- 
TOI,  the  non-TOI  designation  was  considered  correct.  As  discussed  in  Section  3.8,  the  final  list 
of  all  TOI  (including  blind  seed  items)  recovered  during  the  intrusive  investigation  will  be 
compared  to  the  ranked  dig  list  upon  completion  of  the  intrusive  investigation. 

3.9.3  Success  Criteria 

The  perfonnance  objective  was  considered  met  if  none  of  the  seed  items  were  classified  as  non- 
TOI,  and  more  than  75%  of  targets  collected  were  classified  as  non-TOI. 

3.10  CORRECTLY  IDENTIFY  TYPE  OF  TARGET  OF  INTEREST 

In  addition  to  correctly  identifying  each  TOI  as  a  target  that  needs  to  be  intrusively  investigated, 
there  is  value  in  correctly  specifying  what  the  source  is  before  the  intrusive  efforts  commence. 
Knowing  the  type  of  ordnance  potentially  being  excavated  can  have  an  effect  on  the  distances 
used  for  required  road  closures  during  excavation.  Correctly  predicting  sources  also  provides 
confidence  that  the  classification  process  is  working  as  intended. 

3.10.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  TOI  items  that  are  assigned  to  the  correct 
group  in  the  dig  list.  The  groups  are  described  below. 
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3.10.2  Data  Requirements 

Those  targets  identified  as  a  dig  target  on  the  submitted  dig  list  were  labeled  as  a  either  small, 
medium,  or  large  item.  The  small  group  covered  fuze  boosters  and  small  ISOs;  the  medium 
group  covered  75-millimeter  (mm)  projectiles,  60-mm  and  81  -mm  mortars,  2.36-inch  (in.)  and 
2.75-in.  rockets,  and  medium  ISOs;  the  large  group  covered  any  projectiles  larger  than  75  mm 
and  large  ISOs.  3.5-in.  rockets,  which  are  approximately  90  mm  in  diameter,  were  placed  into  a 
medium/large  group  given  their  size  relative  to  the  largest  item  typically  considered  medium  in 
size  (81  mm  mortar)  and  the  smallest  item  typically  considered  large  in  size  (105  mm  projectile). 
Identifications  were  assigned  based  on  the  degree  of  match  to  the  munition  or  simulant  examples 
in  the  classification  library  (Section  6.3). 

3.10.3  Success  Criteria 

The  performance  objective  was  considered  met  if  more  than  75%  of  the  TOI  items  were  placed 
into  the  correct  group. 

3.11  CORRECTLY  PREDICT  ATTRIBUTES  OF  NON-TARGETS  OF  INTEREST 

While  the  classification  of  non-TOI  does  not  generally  affect  intrusive  investigation  operations, 
meeting  the  success  criteria  for  the  correct  prediction  of  non-TOI  attributes  can  play  an  important 
role  in  convincing  stakeholders  of  the  basis  and  justification  for  not  intrusively  investigating 
non-TOI  items,  and  it  addresses  the  potential  concern  of  false  negative  results. 

3.11.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  non-TOI  for  which  at  least  one  attribute  was 
correctly  predicted  by  the  data  analyst. 

3.11.2  Data  Requirements 

Rather  than  predicting  size  and  shapes  for  all  of  the  targets  classified  as  non-TOI,  the  QC 
Geophysicist  randomly  selected  50  targets  classified  as  non-TOI  on  the  final  ranked  dig  list.  The 
data  analyst  gave  brief  explanation  as  to  why  each  of  those  targets  was  classified  as  a  non-TOI. 
Reasonable  explanations  included  descriptions  of  target  characteristics  such  as  response  (too 
small  to  be  TOI),  apparent  shape  (non-symmetric  or  plate-like  rather  than  apparently  cylindrical), 
decay  (rapid  decay  indicates  item  walls  too  thin  to  be  TOI),  or  a  combination  of  these 
descriptions.  Other  reasonable  explanations  were  accepted  as  long  as  they  predicted  an  attribute 
of  the  source  that  could  be  qualitatively  judged  by  the  QC  Geophysicist  based  on  the  photo  of  the 
source  taken  during  the  intrusive  investigation. 

3.11.3  Success  Criteria 

The  project  objective  was  considered  met  if  100%  of  the  non-TOI  recovered  qualitatively 
matched  the  data  analyst’s  predictions.  Because  this  performance  objective  is  entirely  dependent 
on  photos  of  non-TOI  taken  during  the  intrusive  investigation,  there  is  currently  no  way  to 
determine  success  or  failure.  The  analyst  has  provided  justification  for  the  50  non-TOI  anomalies 
selected  by  the  QC  Geophysicist,  but  this  objective  cannot  be  measured  effectively  until  after  the 
intrusive  investigation. 
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3.12  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

This  objective  involves  the  accuracy  of  the  modeled  location  for  target  source  objects.  The 
correct  estimation  of  the  location,  both  horizontally  and  vertically,  increases  confidence  in  both 
the  dig  team  and  stakeholders  that  the  correct  source  object  is  being  investigated  if  the  estimated 
location  is  relatively  close  to  the  recovered  object’s  location. 

3.12.1  Metric 

The  distance  between  the  inverted  target  location  and  the  location  of  the  object(s)  recovered 
during  the  intrusive  investigation  was  the  metric  for  this  objective. 

3.12.2  Data  Requirements 

The  dig  list  submitted  to  the  intrusive  team  contained  X,  Y,  and  Z  locations  for  each  target  as 
determined  during  inversion  of  that  target.  These  were  compared  to  the  locations  of  the  blind 
seed  items  as  measured  during  burial.  Offsets  were  calculated  between  the  predicted  and  known 
locations  of  the  seed  items.  As  for  all  other  classification  objectives  a  full  analysis  of  predicted 
versus  actual  locations  for  excavated  targets  will  be  performed  following  the  intrusive 
investigation. 

3.12.3  Success  Criteria 

The  project  objective  was  considered  met  if  one  standard  deviation  (a)  of  the  distance  between 
the  estimated  X,  Y  locations  and  the  known  seeds  locations  were  within  30  cm  and  the  estimated 
depths  were  within  15  cm. 


3-9 


4.0  SITE  DESCRIPTION 


4.1  SITE  DESCRIPTION 

The  PCD  is  located  in  Pueblo  County,  Colorado,  approximately  15  miles  east  of  the  city  of 
Pueblo.  SWMU  7  encompasses  66  acres  of  land  in  the  northwest  corner  of  the  PCD.  The 
demonstration  will  be  conducted  across  8-m  wide  transects  spaced  evenly  across  the  SWMU. 
The  location  of  the  demonstration  area  and  the  transect  locations  are  shown  on  Figure  4.1. 

4.2  SITE  SELECTION 

This  site  was  chosen  as  one  in  a  series  of  sites  for  demonstration  of  the  classification  process. 
Sites,  including  this  one,  provide  opportunities  to  demonstrate  the  capabilities  and  limitations  of 
the  classification  process  on  a  variety  of  site  conditions.  Further  information  about  ESTCP’s 
classification  program  can  be  found  at  https://www.serdp-estcp.org/Program-Areas/Munitions- 
Response/Land/Live-Site-Demonstrations.  This  site  was  selected  for  the  program  because  of  its 
former  use  as  an  open  burn  area  as  opposed  to  a  firing  range  and  available  funding  for  the 
intrusive  effort  as  part  of  the  Corrective  Measures  Study  Work  Plan  development. 

4.3  BRIEF  SITE  HISTORY 

The  land  for  the  PCD  was  acquired  in  1941  and  1942,  at  which  time  it  was  referred  to  as  the 
Pueblo  Ordnance  Depot.  It  was  administered  by  the  U.S.  Army  Ordnance  Corps  and  mainly  used 
for  ordnance  storage.  The  maintenance  and  repair  of  combat  materials  was  added  to  the  depot’s 
mission  in  the  mid-1940s  as  equipment  returned  from  overseas  at  the  end  of  World  War  II,  and 
the  secure  storage  of  chemical  weapons  began  in  the  1950s  when  chemical  agents  and  munitions 
were  transferred  to  the  depot  from  Rocky  Mountain  Arsenal  in  Denver.  Through  various 
realignments,  the  depot  has  been  referred  to  as  the  Pueblo  Anny  Depot,  Pueblo  Depot  Activity, 
and  the  Pueblo  Chemical  Depot.  The  current  mission  continues  to  the  secure  storage  of  chemical 
weapons  as  a  plant  designed  to  destroy  the  stored  munitions  is  constructed  on  site. 

Between  1953  and  1990,  SWMU  7  was  used  for  the  open  burning  of  munitions,  generally  in 
bum  trays,  although  materials  were  also  burned  in  trenches  in  the  area.  Ash  and  residue  from 
burning  operations  were  reportedly  piled  in  the  southeast  corner  of  the  bum  area.  The  former 
location  of  the  Pyrotechnic  Burning  Cage,  used  for  burning  small  munitions,  rocket  motor 
igniters,  and  fireworks  and  designated  as  SWMU  10,  is  also  completely  within  the  SWMU  7 
boundary.  SWMU  10  will  be  remediated  and  closed  as  part  of  SWMU  7  through  the  Resource 
Conservation  and  Recovery  Act  process. 

A  munitions  removal  project  was  performed  in  SWMU  7  in  1997.  Activities  consisted  of 
performing  a  “mag  and  flag”  clearance  operation  on  72  200-ft.  by  200-ft.  grids  (66  acres)  to  a 
depth  of  1  foot.  Activities  covered  the  entire  open  bum  area,  including  a  buffer  kick-out  area.  Of 
the  22,385  anomalies  intmsively  investigated,  18  live  MEC  items  were  recovered  along  with 
15,592  pounds  of  MD  and  non-MD  scrap.  Recovered  MEC  included  projectiles,  rocket  warheads 
and  motors,  mortars,  boosters,  and  a  small  piece  of  high  explosives.  QA/QC  verification 
procedures  were  only  perfonned  on  14  of  the  72  grids  cleared.  The  remaining  58  grids  were  not 
completed  but  had  various  amounts  of  work  completed.  Field  notes  from  the  project  suggest  that 
heavy  equipment  may  be  needed  to  complete  the  work. 
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4.4  MUNITIONS  CONTAMINATION 

The  suspected  munitions  in  the  SWMU  7  include: 

•  Fuze  boosters 

•  75 -mm  and  larger  projectiles 

•  2.36-in.,  2.75-in.,  and  3.5-in.  rockets 

•  60-inm  and  81 -mm  mortars 

4.5  SITE  GEODETIC  CONTROL  INFORMATION 

Two  first-order  location  control  points  were  installed  by  a  professional  surveyor  licensed  in  the 
State  of  Colorado.  One  was  used  for  placement  of  the  GPS  base  station,  and  the  other  was  used 
for  QC  checks.  The  coordinates  for  the  locations  of  the  first-order  points  used  for  this 
demonstration  are  provided  in  Table  4. 1 . 


Table  4.1 

Geodetic  Control  Locations 


ID 

Northing  (m) 

Easting  (m) 

Ellipsoid  Height  (m) 

ESTCP 1 

4242851.171 

554552.675 

1401.214 

ESTCP2 

4242858.571 

554531.008 

1400.956 

4.6  SITE  CONFIGURATION 

The  location  chosen  for  the  demonstration  site  is  shown  on  Figure  4.1.  A  shapefile  delineating 
the  demonstration  site  boundary  and  a  text  file  of  the  corner  coordinates  are  available  from  the 
ESTCP  Program  Office. 
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Figure  4.1  Proposed  Demonstration  Area 
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5.0  TEST  DESIGN 


5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  objective  of  this  program  is  to  demonstrate  a  method  for  the  use  of  classification  in  the  MR 
process.  The  three  key  components  of  this  method  are  1)  collection  of  high-quality  geophysical 
data  and  principled  selection  of  anomalous  regions  in  those  data;  2)  analysis  of  the  selected 
anomalies  using  physics-based  models  to  extract  target  parameters  such  as  size,  shape,  and 
materials  properties;  and  3)  the  use  of  those  parameters  to  construct  a  ranked  anomaly  list. 

Many  projects  to  date  have  used  the  EM61  metal  detector  for  the  digital  geophysical  mapping 
(DGM)  portion  of  the  project  and  have  collected  cued  MetalMapper  data  over  target  locations 
identified  in  the  EM61  data.  At  PCD,  the  MetalMapper  was  used  for  both  dynamic  and  cued 
data  collection.  An  attempt  was  made  to  use  a  dipole  filter  to  identify  dynamic  anomalies  and 
screen  out  those  too  small  to  be  TOI.  However,  the  extremely  small  size  of  one  of  the  TOI 
known  to  have  been  found  at  SWMU  7,  the  fuze  booster,  and  extremely  high  anomaly  densities 
at  the  site  limited  the  effectiveness  of  this  method.  Therefore,  dynamic  targets  were  selected 
using  response  amplitude  in  much  the  same  way  as  that  any  other  DGM  dataset  would  be 
processed.  The  MetalMapper  was  then  used  to  collect  cued  data  over  the  targets  identified  in  the 
dynamic  data.  Cued  data  were  processed  using  routines  in  the  UX- Analyze  software  package  to 
extract  target  parameters.  These  parameters  were  passed  to  a  classification  routine  that  was  used 
to  produce  a  ranked  anomaly  list. 

The  primary  objective  of  the  demonstration  was  to  assess  how  well  the  ranked  anomaly  list 
succeeded  in  separating  high  confidence  clutter  from  all  other  items.  A  secondary  objective  was 
to  detennine  the  classification  performance  that  could  have  been  achieved  through  a 
retrospective  analysis.  Because  the  intrusive  investigation  of  the  site  is  not  part  of  this  project,  all 
performance  objective  assessment  in  this  report  was  completed  using  the  blind  seeds  as  the  only 
known  TOI. 

5.2  SITE  PREPARATION 

5.2.1  First-order  Navigation  Points 

All  survey  data  and  validation  activities  must  be  conducted  on  a  common  coordinate  system. 
Two  first  order  navigation  points  were  set  at  the  site  by  a  professional  land  surveyor  prior  to  the 
start  of  the  project.  The  coordinates  for  these  points  are  included  in  Table  4. 1 . 

5.2.2  Surface  Clearance 

All  visible  metal  objects  were  removed  from  the  surface  of  the  demonstration  area  prior  to  data 
collection. 

5.2.3  Seeding  Operation 

Typically,  at  a  live  site  such  as  this,  there  is  a  high  ratio  of  clutter  to  TOI  and  only  a  small 
number  of  TOI  are  found  in  the  investigation  to  detennine  classification  performance  with 
acceptable  confidence  bounds.  Additionally,  for  this  project,  the  schedule  for  the  intrusive 
investigation  at  the  site  is  currently  unknown,  and  no  native  TOI  information  will  be  available 
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until  that  is  complete.  Therefore,  the  site  was  seeded  with  enough  TOI  to  ensure  reasonable 
statistics  and  to  determine,  to  some  extent,  the  effectiveness  of  the  project  prior  to  the  intrusive 
investigation. 

Parsons  conducted  seeding  operations  at  PCD  on  August  19,  2015  and  20,  2015.  Industry 
standard  objects  (ISOs)  and  inert  munitions  were  buried  throughout  the  survey  area  prior  to  the 
dynamic  survey.  The  location  of  each  seed  item  was  established  with  a  Trimble  R8  RTK  GPS 
system.  The  base  station  control  point  used  for  this  operation  was  established  prior  to  the  start  of 
this  investigation  using  Universal  Transverse  Mercator  (UTM)  Zone  13N,  WGS84  coordinates. 

Parsons  seeded  40  items  and  practiced  anomaly  avoidance  at  each  location  for  safety  and  to 
ensure  a  clean  area  for  emplacement.  All  40  seeds  were  placed  at  the  planned  depths  and 
orientations.  Excavation  operations  involved  manual  procedures  to  meet  precise  specifications 
and  to  minimize  burial  evidence.  Prior  to  emplacement,  magnetic  north  was  determined.  The 
seed  item  was  positioned  with  the  nose  pointing  to  the  azimuth  and  orientation  planned.  A  photo 
was  taken  of  the  seed  item  at  the  burial  location.  The  seed  item  was  then  photographed  along 
with  a  whiteboard  showing  emplacement  information. 

Seed  location  holes  were  not  backfilled  until  final  QC  checks  were  complete.  QC  checks 
consisted  of  comparing  the  location  with  the  original  designated  location;  capturing  the  center 
location  of  the  emplaced  seed  item  with  GPS;  and  checking  the  depth,  inclination,  and  dip  angle 
of  each  seed  item.  After  these  checks  were  complete,  the  hole  was  backfilled  with  a  shovel  to 
prevent  any  excess  movement  of  the  seed  items.  A  list  of  the  seed  items  emplaced  for  the  project 
is  included  in  Table  5.1. 


Table  5.1 

PCD  Demonstration  Seed  Items 


Seed  item 

Total 

;  Small  ISO 

24 

Medium  ISO 

4 

Large  ISO 

2 

Fuze  booster 

2 

60-mm  mortar 

2 

81 -nun  mortar 

2 

2.36-in.  rocket 

2 

105-mrn  projectile 

2 

Total 

40 
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5.2.4  Establish  an  Instrument  Verification  Strip 

A  relatively  anomaly-free  area  was  identified  adjacent  to  the  storage  connex  just  to  the  southeast 
of  SWMU  7  for  use  as  the  IVS.  The  strip  used  for  the  IVS  was  roughly  15  m  long.  A  list  of  the 
seed  items  placed  in  the  IVS  is  included  in  Table  5.2. 


Table  5.2 
IVS  Seed  Items 


IVS  item 

Description 

Depth  (cm) 

Orientation 

IVS-1 

Small  ISO 

15 

Horizontal 

IVS -2 

Medium  ISO 

25 

Horizontal 

IVS -2 

Large  ISO 

50 

Horizontal 

IVS -4 

Background  location 

N/A 

5.3  SYSTEM  SPECIFICATION 

The  MetalMapper  sensor  and  data  acquisition  system  are  described  in  detail  in  Section  2.1. 
During  the  demonstration,  the  sensor  was  transported  via  a  sled  mounted  to  the  front  of  a  tracked 
skid  steer  (Figure  2.2).  A  Trimble  R8  GPS  was  mounted  directly  above  the  sensor  array  using  a 
wooden  tripod,  and  an  inertial  measurement  unit  was  attached  to  the  wooden  support  used  to 
stabilize  the  X-  and  Y-direction  transmitters,  also  directly  above  the  center  of  the  array.  These 
instruments  streamed  positional  data  constantly  at  a  rate  of  10  Hz.  The  two  instruments  were 
connected  to  the  DAQ  via  USB  (universal  serial  bus)  ports.  Incoming  GPS  data  were  used  to 
navigate  along  predefined  survey  lines  for  the  dynamic  survey  or  from  point  to  point  in  the  cued 
survey  and  to  locate  the  collected  response  data.  IMU  corrections  are  not  perfonned  in  real  time 
and  integrated  with  the  incoming  GPS  data  on  the  MetalMapper  screen,  but  they  were  used  to 
correct  the  locations  of  all  collected  GPS  points  based  on  the  pitch,  roll,  and  yaw  information 
recorded  with  the  GPS  measurements. 

5.4  CALIBRATION  ACTIVITIES 

5.4.1  Instrument  Verification  Strip  Data  Collection 

Data  was  collected  over  the  IVS  twice  daily  during  both  the  dynamic  and  cued  surveys.  All  data 
collected  over  the  IVS  strip  were  processed  as  described  in  Section  6.2  and  compared  to  either 
expected  responses  (dynamic)  or  the  Pueblo  target  library  (cued).  The  following  tests  were 
performed  for  the  collected  IVS  data: 

•  Dynamic:  the  response  measured  for  each  seed  item  was  compared  to  a  minimum 
expected  response  determined  for  that  item.  Unlike  the  EM61,  for  which  there  are 
expected  response  versus  depth  curves  for  standard  seed  and  ordnance  items,  the 
responses  versus  depth  for  the  MetalMapper  are  almost  completely  unknown. 
Therefore,  the  minimum  expected  response  for  each  IVS  seed  was  defined  as  the 
average  response  for  that  item  for  the  first  five  passes  over  the  IVS.  All  IVS 
responses  were  determined  using  the  average  of  the  responses  measured  by  the  five 
middle  receivers  of  the  MetalMapper. 
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•  Cued:  the  item  identified  by  the  target  library  comparison  was  compared  to  the  actual 
buried  item,  and  it  was  expected  that  the  identified  item  matched  the  seed  item  with  a 
relatively  high  confidence  (>  0.90  for  the  pi:p2:p3  confidence  metric).  Identified 
results  were  considered  a  match  to  the  I  VS  seed  as  long  as  the  sizes  of  the  two  items 
were  relatively  similar  (i.e.,  medium  ISO  seed  identified  as  a  2.75-in.  rocket  warhead 
was  acceptable,  medium  ISO  identified  as  a  full  5-in.  rocket  was  not). 

IVS  testing  results  are  detailed  in  Sections  7.1  and  7.5. 

5.4.2  Function  Tests 

Although  not  specified  in  the  demonstration  plan,  the  use  of  function  tests  was  evaluated  during 
the  project.  These  involved  the  collection  of  a  background  point  followed  by  the  collection  of  a 
point  with  ISOs  placed  in  consistent  locations  on  the  Z-transmitter  housing.  Tests  were 
performed  using  both  small  ISOs  and  medium  ISOs  as  the  test  objects  for  different  parts  of  the 
project.  The  results  of  the  function  tests  were  evaluated  by  subtracting  the  background  response 
from  the  test  item  response,  then  comparing  the  absolute  value  of  the  maximum/minimum 
measured  by  each  of  the  2 1  receiver  wrappings  on  or  after  time  gate  8  with  the  average  of  the 
previous  responses  measured  by  each.  Function  tests  were  considered  successful  if  the  measured 
responses  for  each  receiver  wrapping  were  within  20%  of  the  average  response. 

5.4.3  Background  Data 

MetalMapper  background  collection  points  were  pre-selected  based  on  the  dynamic  data. 
Background  data  were  collected  at  one  of  the  pre-selected  locations  at  least  once  every  2  hours. 
Additional  points  were  collected  at  intervals  less  than  2  hours  if  the  operator  felt  another  point 
was  necessary  for  any  reason  (e.g.,  changes  to  the  configuration  of  the  instrument,  changing  field 
conditions  such  as  rain). 

5.5  DATA  COLLECTION  PROCEDURES 

5.5.1  Dynamic  Data 

As  shown  in  Figure  4.1,  nine  8-m  wide  transects  were  established  at  approximately  50-m 
intervals  across  SWMU  7  in  order  to  determine  relative  anomaly  densities  throughout  the  site. 
They  were  identified  as  Transects  1  through  9,  starting  with  the  very  small  transect  on  the  south 
end  of  the  site  with  identification  (ID)  number  increasing  to  the  north.  Dynamic  MetalMapper 
data  were  collected  within  these  transects  by  driving  east-west  along  parallel  lines.  Although  the 
dynamic  footprint  of  the  MetalMapper  is  assumed  to  be  70  cm  (Section  3.2),  it  is  difficult  to 
keep  the  sensor  exactly  aligned  on  the  intended  transect  because  it  is  so  far  in  front  of  the 
transport  vehicle.  Therefore,  survey  lines  were  collected  at  50-cm  spacings  to  ensure  full 
coverage  of  the  survey  areas. 

The  MetalMapper  had  a  GPS  antenna  directly  over  the  center  of  the  sensor,  which  transmitted 
real-time  positioning  data  to  the  data  acquisition  computer  DAQ.  The  location  of  the  instrument 
during  collection  was  displayed  on  the  DAQ  screen  along  with  the  predetennined  locations  of 
the  data  lines  to  be  collected.  The  operator  drove  at  a  speed  of  less  than  1  meter  per  second, 
keeping  the  instrument  centered  on  the  selected  line  with  the  aid  of  a  light  bar  at  the  top  of  the 
DAQ  screen  that  indicated  when  the  GPS  deviated  from  the  predetennined  path  and  in  which 
direction  the  deviation  occurred.  IMU  data  were  collected  simultaneously  with  the  MetalMapper 


5-4 


data,  although  the  data  from  this  instrument  are  not  integrated  in  real  time  on  the  screen.  The 
IMU  data  were  used  in  post-processing  to  correct  the  sensor  location  for  pitch,  roll,  and  yaw 
variations  due  to  terrain.  These  corrections  were  very  important  at  PCD  because  the  site 
contained  numerous  drainage  ditches  that  caused  significant  pitch  and  roll  changes  during  the 
survey.  The  MetalMapper  data  acquisition  parameters  for  the  dynamic  survey  are  contained  in 
Table  5.3. 


Table  5.3 


Dynamic  Data  Acquisition  Parameters 


Mode 

Tx 

Mode 

Hold-Off 
Time  (us) 

Bloek 

Period(s) 

Rep 

Fetr 

Dee 

Fetr 

(%) 

Stk 

Const 

Base 

Freq 

(Hz) 

Decay 
Time  (us) 

No. 

Gates 

Sample 
Period  (s) 

Sample 
Rate  (S/s) 

Dynamic 

Z 

50 

0.1 

9 

20 

1 

90 

2776 

19 

0.1 

10 

5.5.2  Cued  Data 

The  operator  moved  the  array  between  targets  by  lifting  the  sled,  navigating  to  the  vicinity  of 
each  selected  point  using  the  graphic  display  on  the  computer  monitor,  and  sehing  the 
MetalMapper  down  on  the  point.  Reacquisition  of  the  dynamic  targets  selected  for  cued  data 
collection  was  accomplished  using  “dancing  arrows”  displayed  on  the  monitor.  The  dancing 
arrows  display  shows  the  seven  receivers  in  the  array,  arranged  as  they  are  in  the  Z-coil,  typically 
with  a  blue  arrow  pointing  out  of  each.  The  arrows  point  toward  the  metallic  source  nearest  each 
of  the  receivers.  Under  ideal  conditions,  there  is  one  source  in  the  vicinity  of  the  selected  point, 
and  all  of  the  arrows  point  inward  toward  the  center  of  the  array.  In  the  case  of  multiple  sources, 
one  or  more  of  the  outer  arrows  may  point  outward  from  the  array  toward  another  piece  of  metal. 
Generally,  the  operator  attempted  to  position  the  array  such  that,  at  least,  the  arrows  in  the  three 
receivers  closest  the  middle  of  the  coil  were  pointing  at  each  other.  Once  the  MetalMapper  was 
positioned  correctly  above  the  target,  the  operator  collected  a  data  point  using  the  settings 
indicated  in  Table  5.4. 


Table  5.4 

Cued  Data  Acquisition  Parameters 


Mode 

Tx 

Mode 

Hold-Off 
Time  (us) 

Block 

Period(s) 

Rep 

Fetr 

Dec 

Fetr 

(%) 

Stk 

Const 

Base 

Freq 

(Hz) 

Decay 
Time  (us) 

No. 

Gates 

Sample 
Period  (s) 

Sample 
Rate  (S/s) 

Static 

ZYX 

50 

0.9 

27 

10 

10 

30 

8328 

50 

9 

N/A 

Static  targets  were  identified  according  to  the  ID  determined  for  each  target  picked  in  the 
dynamic  survey.  The  transect  locations  of  dynamic  targets  were  identified  in  the  target  number 
by  the  first  digit.  For  example,  all  Transect  3  targets  were  between  30,000  and  39,999;  all 
Transect  4  targets  were  between  40,000  and  49,999,  etc. 

5.5.3  Scale  of  Demonstration 

A  total  of  5.4  acres  of  dynamic  data  collection  was  performed  in  Transects  2,  3,  4,  5,  8,  and  9. 
Given  extremely  high  anomaly  densities  at  the  she,  tens  of  thousands  of  anomalies  were  selected 
in  the  collected  data.  However,  many  of  these  were  within  areas  of  the  she  where  the  analyst 
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considered  anomaly  densities  to  be  too  high  for  either  the  accurate  identification  and  selection  of 
individual  targets  potentially  representative  of  TOI  as  small  as  fuze  boosters  or  the  accurate 
classification  of  collected  cued  targets  given  the  small  size  of  the  fuze  boosters  and  the 
abundance  of  subsurface  sources.  While  targets  were  auto-picked  using  the  Blakely  test 
algorithm  within  these  high-density  areas  (HD;  Section  6.1),  they  were  generally  not  given  target 
IDs  because  target  selection  was  not  considered  effective  within  these  areas.  No  targets  were 
picked  in  Transect  5  because  the  analyst  felt  that  most  of  this  transect  would  be  considered  an 
HD  area.  A  total  of  14,215  individual  targets  were  selected  and  given  target  IDs,  most  of  these 
outside  of  the  analyst-defined  HD  areas,  although  some  targets  within  these  areas  were  retained 
at  the  discretion  of  the  QC  Geophysicist. 

Out  of  the  14,215  transect  targets,  1,978  were  selected  by  the  QC  Geophysicist  and  placed  on  the 
cued  target  list.  Based  on  available  time  and  budget,  a  total  of  1,164  cued  targets  were  collected 
out  of  the  1,978  on  the  cued  list. 

5.5.4  Sample  Density 

One  data  point  was  collected  per  target,  as  described  in  Section  5.5.  Based  on  available  time  and 
budget,  re-shots  were  not  collected  for  cued  targets  that  exceeded  either  collected  location  to 
modeled  location  (Section  3.6)  or  dynamic  target  location  to  collection  location  (Section  3.7) 
metrics.  Targets  exceeding  these  metrics  for  which  re-shots  typically  would  have  been 
performed  were  given  their  own  category  on  the  final  dig  sheet  to  reflect  their  status  as  collected 
but  requiring  uncollected  re-shots  (Section  6.2) 

5.5.5  Data  Quality  Checks 

I  VS  data  collection  and  function  tests  were  conducted  twice  a  day  (at  the  beginning  and  the  end 
of  the  field  day).  These  checks  ensured  that  the  instrumentation  was  functional,  properly 
calibrated,  and  stable. 

Checks  on  the  quality  of  both  dynamic  and  static  data  were  performed  during  processing  using 
the  UX-Detect  and  UX- Analyze  modules  in  Oasis  montaj.  Dynamic  data  were  checked  for 
spatial  coverage  and  down-line  point  spacing;  cued  data  were  checked  for  offsets  between  the 
collection  location  and  modeled  source  location  and  the  collection  location  and  the  dynamic 
target  location.  A  detailed  discussion  of  these  checks  is  contained  in  Section  3. 

5.5.6  Data  Handling 

Data  were  recorded  in  binary  fonnat  as  files  on  the  hard  disk  of  the  MetalMapper  DAQ.  These 
data  were  offloaded  to  other  media  at  least  once  per  day.  The  DAQ  hard  disk  had  enough 
capacity  to  store  all  the  data  from  the  entire  site,  so  these  data  were  not  erased  until  they  had 
been  thoroughly  reviewed  and  archived.  The  data  file  names  acquired  each  day  were  cataloged 
and  integrated  with  any  notes  or  comments  in  the  operator’s  field  book.  All  data  ended  up  on  the 
hard  drives  of  one  or  more  laptop  computers  used  to  post-process  data.  Data  were  also  archived 
to  a  data  server  in  the  Parsons  office. 

Raw  binary  files  were  preprocessed  using  the  TEM2CSV  software  package,  which  outputs 
“preprocessed  and  located”  data  files  in  a  text  readable  fonnat  (.CSV).  Preprocessing  included 
the  location  of  the  point  in  UTM  meters.  Located  and  background  corrected  .CSV  files  were 
imported  into  Oasis  montaj  for  further  processing  and  analysis. 
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6.0  DATA  ANALYSIS  AND  PRODUCTS 


The  MetalMapper  was  used  to  collect  dynamic  data  over  5.4  acres  and  static  data  over  1,164 
targets  identified  in  the  dynamic  survey.  The  processing  and  analysis  steps  that  were  used  to 
generate  a  dig/no  dig  decision  for  each  target  are  described  below. 

6.1  DYNAMIC  DATA 

6.1.1  Preprocessing  and  Processing 

Raw  MetalMapper  data,  both  dynamic  and  static,  are  collected  and  stored  as  .tern  files.  The 
MetalMapper  acquisition  software  uses  a  convention  for  assigning  a  unique  name  to  each  data 
file  without  the  need  to  manually  enter  the  name.  The  operator  supplies  a  prefix  for  the  root 
name  of  the  file  (e.g.,  “Dyn”  or  “Stat”).  The  acquisition  software  then  automatically  appends  a 
five-character  numerical  index  to  the  filename  prefix  to  fonn  a  unique  root  name  for  the  data  file 
(e.g.,  DynOOOOl).  The  index  is  automatically  incremented  after  the  file  has  been  successfully 
written.  Each  dynamic  survey  line  was  stored  as  a  separate  file.  Preprocessing  of  the  .tern  files 
was  accomplished  using  TEM2CSV,  a  program  specifically  developed  for  this  purpose.  Very 
little  preprocessing  was  done  to  the  dynamic  data  using  TEM2CSV  aside  from  refonnatting  the 
binary  data  to  a  file  structure  that  could  be  imported  into  Geosoft. 

Dynamic  data  processing  and  target  selection  were  primarily  performed  using  the  UX-Analyzc 
module  in  Geosoft’s  Oasis  montaj.  Upon  import  into  Geosoft,  all  receiver  data  are  given  the 
same  position  (i.e.,  the  position  of  the  GPS  during  collection  of  each  data  point),  which  has  not 
been  corrected  based  on  the  IMU  data.  These  single  positions  were  used  to  calculate  the  survey 
coverage  and  down-line  point  spacing  metrics.  Once  these  QC  checks  were  completed,  the  data 
recorded  by  each  individual  receiver  were  positioned  according  to  where  the  receivers  actually 
were  relative  to  the  position  reported  by  the  GPS.  In  addition  to  splitting  out  the  data  measured 
by  individual  receivers,  position  corrections  were  applied  based  on  collected  IMU  data. 

The  dipole  response  filter  approach  was  applied  to  a  subset  of  the  collected  data  in  an  attempt  to 
evaluate  its  effectiveness  in  selecting  targets  relative  to  the  traditional  amplitude  response 
method.  This  approach  differs  from  the  traditional  approach  in  that  it  uses  all  available  receive 
channel  data  to  make  the  decision  on  whether  a  viable  target  exists  and  should  be  selected  for 
cued  surveying.  The  dipole  response  filter  method  is  a  two-step  process  that  begins  by 
identifying  all  possible  source  locations,  followed  by  taking  data  about  each  of  these  locations 
and  performing  1-,  2-  &  3-dipole  source  model  inversions.  If  everything  is  consistent,  all  non¬ 
overlapping  identified  source  locations  represent  legitimate  targets  for  possible  selection. 
Unfortunately,  due  to  the  extremely  small  size  of  the  fuze  boosters  (TOIs  known  to  have  been 
found  at  SWMU  7)  and  extremely  high  anomaly  densities  at  the  site,  the  desired  consistency  was 
very  hard  to  achieve  making  the  amplitude  response  method  the  better  choice  for  target  selection. 

Because  of  the  relatively  small  size  of  the  fuze  boosters  suspected  to  be  present  on  site,  only  the 
first  time  gate  measured  by  the  MetalMapper,  time  gate  5,  was  used  for  target  selection.  Time 
gate  5  data  were  exported  from  the  Z-loop  response  array  channel  for  each  receiver.  The 
exported  channel  was  then  leveled  using  a  100-point  rolling  median  filter  that  ignored  10%  of 
the  lowest  responses  within  the  window  and  40%  of  the  highest  responses  within  the  window. 
The  data  measured  by  the  two  outer  receivers,  contained  in  the  L[LineNumber].l  and 
L[LineNumber].7  lines  in  the  Geosoft  databases,  were  unselected  so  they  would  not  be  used 
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during  gridding.  The  leveled  data  were  then  gridded  and  displayed  on  the  site  map.  A  latency 
correction  was  applied,  as  necessary,  to  remove  any  chevron  patterns  evident  in  the  dataset. 

Figure  6.1  shows  the  site-wide  results  of  the  dynamic  surveys.  As  shown,  there  are  some  areas, 
outlined  in  red,  where  the  analyst  felt  that  high  anomaly  density  prevented  the  selection  of 
individual  target  locations  from  being  a  viable  means  of  completely  removing  any  existing  MEC 
hazard.  These  areas  were  delineated  prior  to  target  selection.  No  effort  was  made  to  select  every 
peak  in  these  areas  during  target  selection,  and  very  few  of  the  auto-picked  anomalies  within 
these  areas  were  added  to  the  cued  target  list. 


Figure  6.1  Site-Wide  Dynamic  Data  Results 


Figure  6.1  shows  gridded  results  for  leveled  channel  5  data  measured  by  Z-loop  of  the  5  middle  receivers.  Data 
displayed  using  a  linear  scale  from  -2.5  mV/A  to  5.0  mV/A.  Anomaly  densities  within  the  areas  outlined  in  red  were 
considered  too  high  for  the  effective  selection  of  individual  anomalies.  Most  of  Transect  5  was  considered  high 

density,  and  no  targets  were  selected  on  this  transect. 


6-2 


6.1.2  Target  Selection 

Targets  were  selected  using  the  Blakely  test  algorithm  in  the  UX -Detect  package  in  Oasis  montaj 
with  a  threshold  of  1.8  millivolts  per  ampere  (mV/A),  which  was  5  times  the  background  value 
calculated  over  an  anomaly-free  area  in  the  northeast  comer  of  Transect  3  using  the  subwindow 
statistics  tool  in  UX -Detect.  Five  times  background  was  used  as  the  target  selection  threshold 
because  there  is  no  expected  penetration  or  detection  depth  for  a  fuze  booster,  which  was 
considered  the  most  difficult  to  detect  TOI  at  the  site.  A  signal  to  noise  ratio  of  5  is  typically 
considered  low  enough  to  detect  MEC  to  a  reasonable  depth  compared  to  the  size  of  the  item 
while  also  limiting  false  positives  caused  by  selecting  targets  on  geophysical  noise. 

The  Blakely  test  selections  were  then  reviewed  by  the  processor,  who  merged  target  picks  on  the 
same  anomaly  and/or  added  picks  to  unpicked  peaks  evident  on  the  color-shaded  grid  map. 
After  the  target  list  for  each  transect  was  finalized,  a  footprint  size  was  calculated  for  each  using 
the  Calculate  Signal  Strength,  SNR,  and  Size  option  in  UX-Detect.  For  most  of  the  transects, 
targets  with  an  areal  extent  of  less  than  0.2  m  were  removed  from  consideration  as  potential  TOI 
and  were  not  added  to  the  list  of  targets  to  be  further  investigated  in  the  cued  survey.  The  0.2  nr 
threshold  was  developed  after  cued  data  collection  began  on  Transect  3  due  to  a  large  number  of 
Transect  3  cued  points  for  which  inversion  results  returned  either  no  real  identifiable  source  (i.e., 
fit  locations  in  what  appeared  to  be  background  areas  in  the  dynamic  data)  or  an  extremely  small 
source.  Given  a  limited  number  of  cued  points  to  be  collected  during  the  project,  the  QC 
Geophysicist  detennined  that  a  0.2-m2  size  threshold  would  limit  the  number  of  cued  points 
collected  over  targets  extremely  unlikely  to  be  TOI  while  retaining  all  of  the  seeds  over  which 
dynamic  data  had  been  collected.  Because  cued  data  collection  started  on  Transect  3  before  the 
threshold  was  implemented,  there  are  a  number  of  cued  results  for  targets  with  areal  extents  less 
than  0.2  m2  on  that  transect. 

6.1.3  Results 

Including  auto-picked  targets  within  the  analyst-defined  HD  areas,  tens  of  thousands  of  targets 
were  identified  in  the  dynamic  data.  For  the  most  part,  only  targets  outside  of  the  HD  Area  were 
considered  usable  for  the  purposes  of  building  a  target  list  for  subsequent  cued  data  collection. 
A  total  of  14,215  individual  targets  were  selected  and  given  target  IDs,  most  of  these  outside  of 
the  analyst-defined  HD  areas.  A  few  of  the  targets  with  the  HD  areas  were  selected  for  use  by 
the  QC  Geophysicist  because  they  either  were  a  seed  item  or  were  near  a  seed  item.  Targets 
surrounding  seed  items  in  HD  areas  were  selected  so  the  target  that  was  the  seed  was  not  readily 
apparent  to  either  the  cued  data  collection  team  or  the  cued  data  analyst.  The  final  list  of  targets 
considered  potential  targets  for  cued  data  collection  was  compared  to  the  seed  items  over  which 
dynamic  data  were  collected.  All  covered  seeds  outside  of  the  HD  areas  were  successfully 
detected  within  40  cm  of  the  locations  measured  during  emplacement. 

Out  of  the  14,215  transect  targets,  1,978  were  placed  on  the  cued  target  list.  These  were  selected 
by  the  QC  Geophysicist  based  on  the  following  factors: 

•  Spatially  representative  of  the  entire  site 

•  Included  all  seed  items  outside  of  HD  area  and  a  few  within  these  areas 

•  Ease  of  cued  collection  (targets  selected  based  on  spatial  proximity  outside  of  HD 
areas) 
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In  addition  to  the  primary  objective  of  identifying  targets  for  the  cued  survey,  the  dynamic  data 
were  used  to  determine  the  average  anomaly  density  for  the  non-HD  areas  of  the  site  where 
targets  were  picked  for  possible  inclusion  in  the  cued  survey.  The  anomaly  density  calculation 
included  1.03  acres  of  area  outside  of  the  HD  areas,  which  contained  1,356  targets  with  sizes 
greater  than  0.2  m  .  That  would  equate  to  an  anomaly  density  of  1,316  anomalies  per  acre. 
However,  retrospective  analysis  of  the  selection  threshold  was  performed  by  examining  the 
measured  responses  for  the  two  fuze  booster  seeds  and  seven  deep  (0.25-m  depth)  small  ISO 
seeds  covered  by  the  dynamic  survey.  The  lowest  response  measured  for  any  of  these  items  was 
7.5  mV/A.  A  new  theoretical  selection  threshold  of  6.78  mV/A  was  calculated  by  subtracting  2 
times  the  0.36  mV/A  noise  value  measured  at  the  site  from  the  lowest  response  measured  for  the 
seed  items.  Using  6.78  mV/A  as  the  selection  threshold  resulted  in  994  targets,  for  an  anomaly 
density  of  965  anomalies  per  acre.  A  basic  estimate  of  area  not  saturated  with  anomalies  in 
SWMU  7  suggests  that  the  MetalMapper  could  be  used  effectively  over  at  least  12.5  acres  of  the 
site.  Based  on  the  use  of  the  6.78  mV/A  threshold,  this  area  would  include  approximately  12,000 
targets. 

6.2  CUED  DATA 

6.2.1  Preprocessing 

Raw  cued  fdes  were  stored  as  described  in  Section  6.1.1,  with  each  cued  target  stored  as  a 
separate  file.  Although  the  cued  target  ID  was  not  used  as  the  file  name  in  the  .tern  file,  the 
target  ID  was  stored  in  the  file  according  to  name  of  the  target  highlighted  on  the  MetalMapper 
screen  during  collection.  TEM2CSV  was  also  used  to  preprocess  the  cued  data  files.  In  this 
case,  TEM2CSV  converted  the  points  from  the  geographic  coordinate  system  used  for  collection 
to  the  UTM  Zone  13N  coordinate  system  used  for  processing  and  corrected  the  collection 
location  based  on  the  IMU  data  collected  with  the  MetalMapper  data  in  addition  to  the  same  type 
of  reformatting  performed  for  the  dynamic  data. 

Background  correction  of  the  cued  points  was  performed  in  Geosoft.  The  first  step  was  a  review 
of  each  day’s  background  points  when  compared  to  the  background  dataset  as  a  whole.  Geology 
across  the  site  was  not  expected  to  be  particularly  variable,  so  it  was  assumed  that  any 
background  points  significantly  different  than  the  others  in  the  set  were  due  to  either  a  point 
collected  over  a  small  piece  of  subsurface  metal  not  noticed  during  collection  or  faulty 
equipment.  Each  day’s  background  points  were  also  compared  to  each  other  by  reviewing  the  63 
decays  (three  transmitters  and  seven  tri-axial  receivers)  for  each  point.  As  with  the  comparison 
to  the  full  site  background  dataset,  it  was  not  expected  that  background  values  should  vary 
significantly  through  the  day.  Any  background  points  showing  significant  variability  from  either 
the  site  wide  background  dataset  or  in-day  dataset  were  not  used  to  correct  any  of  the  cued  target 
data  points.  A  list  of  background  points  not  used  was  maintained  in  the  project’s  Microsoft 
Access  database. 

6.2.2  Parameter  Estimation 

All  MetalMapper  data  points  were  inverted  using  UX-Analyze  to  detennine  a  number  of 
modeled  parameters  for  each  target.  These  parameters  included  the  location,  size,  and 
orientation  of  the  source  object;  the  polarizability  of  each  axis  of  the  object;  and  information 
regarding  the  quality  of  the  data  and  the  relative  match  between  the  inverted  data  and  the 
expected  model. 
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All  target  inversion  was  performed  using  the  UX-Analyze  batch  processing  mode,  which 
included  both  single  object  solver  and  multiple  object  solver  inversions.  The  multiple  object 
solver  returns  seven  results  for  each  target  based  on  different  algorithms  used  to  group  the  point 
cloud  generated  during  inversion  into  one  or  more  sources.  The  version  used  for  each  target  was 
the  one  that  had  the  highest  (31:P2:P3  confidence  metric  match  to  an  item  in  the  classification 
library  (Section  6.2.3). 

6.2.3  Confidence  Metrics 

The  polarizability  curves  developed  for  each  target  were  compared  to  a  library  of  known 
polarizability  curves  compiled  using  previous  test  stand  data  and  known  TOIs  from  other  sites. 
The  items  in  the  PCD  comparison  library  were  fuze  boosters,  75-mm  projectiles,  105-mm 
projectiles,  155-mm  projectiles,  60-mm  mortars,  81  -mm  mortars,  2.36-in.  rockets.  2.75-in. 
rockets,  3.5-in.  rockets,  and  separate  components  of  those  rockets  (i.e.,  motors  and  warheads),  as 
available. 

An  initial  comparison  between  the  measured  targets  and  the  library  data  was  perfonned  using  a 
111  confidence  metric  for  the  three  primary  polarizabilities  (size:  1,  shape  1:  1,  shape  2:  1). 
During  this  comparison,  the  seven  results  from  the  multiple  object  solver  were  compared  to  the 
library,  and  the  one  with  the  highest  confidence  metric  was  selected  for  use  with  that  target.  If 
the  result  selected  was  not  the  one  already  in  the  target  database,  the  database  result  was 
replaced.  All  further  confidence  metrics  were  generated  using  the  multiple  object  result  selected 
during  the  weighted  metric  comparison.  In  addition  to  the  111  confidence  metric  generated 
during  the  initial  comparison  of  the  results  to  the  library,  three  more  metrics  were  generated  for 
each  target: 

1)  110  metric  -  size:  1,  shape  1:1,  shape  2:  0 

2)  011  metric  -  size:  0,  shape  1:1,  shape  2:  1 

3)  100  metric  -  size:  1,  shape  0:1,  shape  2:  0 

Polarizability  maps  were  generated  for  all  results  (single  object  solver  and  all  multiple  object 
solver  results)  for  each  target.  These  maps  displayed  the  inverted  polarizability  curves  for  each 
source  and  the  polarizability  curves  for  the  two  closest  library  matches  for  the  111,  110,  and  100 
comparisons  to  the  library. 

6.2.4  Cluster  Analysis 

A  cluster  analysis  was  perfonned  one  the  entire  cued  data  set  using  the  “Perform  self  match  / 
identify  clusters”  and  “Identify  similar  items”  tools  in  the  UX-Analyze  module.  This  analysis 
was  run  using  a  0.95  cluster  threshold  and  required  a  minimum  of  3  similar  objects  to  define  a 
cluster.  A  total  of  seven  clusters  were  identified,  one  of  which  was  clearly  composed  of  small 
ISOs  based  on  the  confidence  metrics  for  the  sources  in  the  cluster.  Typically,  clusters  of  self¬ 
similar  sources  would  be  considered  as  potential  sources  of  training  digs  on  a  project.  In  this 
case,  the  seven  non-small  ISO  clusters  were  all  composed  of  sources  smaller  and  more  thin- 
walled  than  a  fuze  booster  based  on  the  sizes  and  decays  calculated  for  the  sources  in  those 
groups.  It  is  possible  that  these  are  similar  sources,  such  as  rocket  fins  or  some  other  small 
standard  piece  of  a  munition.  However,  because  TOI  smaller  than  a  fuze  booster  is  not  expected 
at  the  site,  the  identified  clusters  are  not  considered  indicative  of  an  unexpected  TOI  and  no 
training  digs  have  been  identified. 
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The  comparison  of  the  full  polarizability  results  for  each  modeled  source  to  the  full  results  for  all 
other  modeled  sources  in  the  dataset  is  considered  a  much  more  comprehensive  comparison  than 
using  the  decay  measured  between  two  specific  times  within  the  full  polarizability  curve,  but  the 
dataset  was  also  evaluated  using  a  parameter  space  plot  that  graphed  each  target  using  a  decay 
(time  gates  8-36)  versus  size  comparison  (Figure  6.2).  As  indicated  in  the  plot,  there  are  few 
obvious  clusters  of  large,  relatively  slowly  decaying  objects  in  the  dataset.  The  locations  of  three 
of  the  non-small  ISO  clusters  identified  using  UX-Analyze  are  shown  in  the  bottom  left  hand 
comer  of  the  figure;  the  other  three  are  outside  of  the  displayed  range  of  the  plot  (i.e.,  sizes  less 
than  -1.2). 

Colors  were  used  to  differentiate  between  targets  classified  as  digs  based  on  the  analyst’s  initial 
review  of  the  polarizability  maps  (Section  6.2.5)  and  the  remaining  non-digs.  The  analyst  first 
rechecked  the  polarizability  curves  for  all  non-digs  within  areas  occupied  by  digs  to  ensure  that 
they  were  not  just  misclassified  during  the  initial  classification  process.  Any  targets  having 
polarizability  curves  generally  indicative  of  a  cylindrical  object  (pi  greater  than  roughly  equal  P2 
and  P3  curves)  were  added  as  digs.  Remaining  non-digs  within  dig  areas  were  non-symmetric  or 
plate-like  objects  that  happened  to  have  decay/size  characteristics  similar  to  ordnance. 

6.2.5  Classification  Decisions 

The  first  step  in  the  classification  process  was  a  full  review  of  the  offsets  between  the  collection 
point  and  modeled  source  locations  and  the  collection  point  and  the  dynamic  target  location. 
Any  targets  with  no  sources  within  40  cm  of  the  collection  point  were  identified  as  requiring  a 
re-shot  on  the  modeled  location  of  the  single  object  solver  source  (“RS  FIT”  in  the 
UXACOMMENTS  column  of  the  target  database),  and  any  targets  without  a  collection  point 
within  40  cm  of  the  dynamic  target  were  identified  as  requiring  a  re-shot  directly  over  the 
dynamic  target  (“RS  DYN”  in  the  UXA_COMMENTS  column  of  the  target  database).  Time 
and  budget  constraints  prevented  the  collection  of  any  re-shots  during  this  project,  so  all  targets 
requiring  re-shots  could  not  be  classified  effectively.  These  were  placed  in  their  own  category 
on  the  final  ranked  dig  list. 

Once  targets  requiring  re-shots  were  identified,  the  analyst  performed  a  full  review  of  the 
polarizability  maps  created  for  each  modeled  source  in  the  dataset.  This  review  consisted  of 
identifying  targets  that  appeared  to  be  TOI,  with  a  “dig”  note  generally  added  to  the 
UXA  COMMENTS  column  based  on  the  decision  logic  shown  in  Table  6.1.  The  analyst  also 
added  notes  for  targets  that  were  close  to  passing  the  metrics  in  Table  6.1  but  were  noisy  due  to 
the  depth  of  the  source  or  were  affected  by  nearby  sources,  particularly  in  HD  areas.  Finally,  the 
analyst  also  added  notes  for  sources  that  should  be  ignored  during  further  analysis  (i.e.,  space 
plot  analysis,  when  deciding  which  source  would  be  chosen  as  the  final  representative  for  a 
target).  Sources  the  analyst  felt  could  safely  be  ignored  included: 

•  Multiple  object  solver  results  modeling  outside  of  the  sensor,  particularly  if  the 
modeled  source  appeared  to  be  caused  by  an  adjacent  dynamic  target 

•  Sources  modeling  at  significant  depth  where  the  dynamic  results  do  not  indicate  the 
likely  existence  of  a  large,  deep  source  (typically  an  indication  of  background  being 
modeled) 

•  Sources  indicative  of  likely  TOI  for  which  the  analyst  felt  another  modeled  source 
for  that  target  was  a  better  representation  of  the  expected  TOI,  based  either  on  TOI 
size  or  the  modeled  location  of  the  source. 
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Figure  6.2 

Parameter  Space  Plot  (Decay  8-36)  Versus  Size 
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All  “ignore”  notes  included  a  justification  as  to  why  the  analyst  felt  that  source  should  be 
excluded  from  further  analysis.  The  space  plot  analysis  described  in  Section  6.2.4  was 
performed  after  the  initial  review  of  the  polarizability  maps.  Once  the  full  initial  review  of  all 
sources  was  complete,  a  single  source  was  chosen  to  represent  each  target  location.  Typically 
this  was  the  source  with  the  highest  111  confidence  metric,  although  the  final  decision  was  also 
influenced  by  the  analyst’s  notes. 

Table  6.1 


Decision  Logic  Used  for  Initial  Target  Review 


Possible 

TOI? 

111 

confidence 

metric 

110 

confidence 

metric 

100 

confidence 

metric 

Comment 

Yes 

>0.70 

>0.75 

>0.80 

All  three  are  true  [ 

Yes 

NA 

>0.75 

>0.80 

One  non-pi  polarizability  curve  identified  as 
potentially  unusable  by  analyst;  other  two 
true  for  usable  curves 

Yes 

NA 

NA 

>0.80 

Two  non-pi  polarizability  curves  identified  as 
potentially  unusable  by  analyst;  pi  curve 
looks  like  TOI 

No 

<0.60 

<0.70 

<0.80 

Any  one  of  three  is  true  for  polarizability 
curves  considered  usable  by  analyst 

Although  the  analyst’s  initial  impressions  regarding  the  likelihood  that  a  particular  source  were 
recorded  during  the  initial  target  review,  most  classification  was  performed  using  the  “Classify 
and  Rank”  module  built  into  UX-Analyze.  This  module  can  automate  a  significant  amount  of 
the  processing  performed  for  classification  data,  including  inversion;  library  matching;  cluster 
analysis;  classification  of  all  sources  as  TOI,  non-TOI,  or  Inconclusive;  and  the  selection  of  a 
single  source  to  represent  each  target.  All  of  the  cued  data  collected  during  the  project  were  run 
through  the  entire  process  using  this  module  separately  from  the  initial  review  performed  by  the 
analyst.  There  are  a  number  of  thresholds  that  can  be  set  by  the  analyst  within  the  module,  but 
the  most  important  in  the  classification  process  are  the  decision  metric  thresholds  used  to  classify 
and  categorize  each  source.  The  decision  metric,  calculated  automatically,  is  an  average  of  the 
four  confidence  metrics  (111,  110,  Oil,  and  100)  determined  during  the  comparison  of  the 
source  to  the  library.  The  analyst  used  the  default  thresholds  in  the  “Set  thresholds  and 
prioritize”  tool  within  the  Classify  and  Rank  module: 

•  Boundary  of  buffer  and  TOI:  0.925  (category  1) 

•  Boundary  of  buffer  and  non-TOI:  0.825  (category  2) 

•  Boundary  of  buffer  and  non-TOI  weak  targets:  0.750  (category  2),  with  weak  targets 
defined  as  those  with  a  signal  strength  less  than  20 

•  High  confidence  match  to  clutter:  Not  applicable,  no  comparison  to  known  clutter 

As  with  the  earlier  initial  review,  this  analysis  returned  a  single  classification  result  for  each 
target.  The  classify  and  rank  results  were  compared  to  the  analyst’s  initial  review  for  each  target 
to  identify  those  targets  for  which  the  automated  classification  performed  in  UX-Analyze  might 
have  missed  something  noted  by  the  analyst,  including: 

•  Targets  classified  as  TOI  based  on  the  analyst’s  assessment  that  one  or  more  of  the 
polarizability  curves  was  poor 
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•  Noisy  data  for  a  potentially  deep  target 

•  Source  results  potentially  affected  by  other  nearby  sources 

•  Targets  classified  as  non-digs  (not  necessarily  non-TOI)  because  the  source  modeled 
for  that  target  was  the  same  source  modeled  for  another  target,  which  was  common 
for  larger  dynamic  anomalies  with  multiple  dynamic  targets  picked  within  their 
boundaries 

•  Targets  classified  as  Inconclusive  based  on  comparison  of  the  polarizability  curves 
to  the  dynamic  anomaly  that  were  not  classified  as  such  automatically. 

Classification  was  considered  complete  once  the  classify  and  rank  results  were  rectified  with  the 
analyst’s  notes. 

6.3  RANKED  DIG  LIST 

All  collected  targets  were  ranked  according  to  category  and  decision  statistic.  The  categories 
used  for  the  project  are  summarized  in  Table  6.2,  and  the  decision  statistics  used  were  the 
average  of  the  four  confidence  metrics  calculated  during  library  matching.  Categories  were 
sorted  from  low  to  high,  and  targets  within  each  category  were  sorted  by  descending  decision 
statistic.  Targets  classified  as  TOI  were  given  a  predicted  size,  as  described  in  Section  3.10, 
based  on  the  library  object  with  the  best  111  confidence  match  to  the  source  in  question.  If  a 
target  was  classified  as  TOI  using  data  with  fewer  than  three  usable  polarizability  curves,  the  size 
was  determined  using  the  110  or  100  match  as  applicable.  A  brief  summary  of  the  decisions 
made  on  the  final  dig  list  are  contained  in  Table  6.3.  Note  that  there  are  1,165  records  in  the  dig 
list  for  1,164  targets.  It  contains  two  entries  for  target  42598.  One  of  these  is  a  non-TOI 
classification  based  on  a  cued  point  collected  8  cm  from  the  dynamic  target  and  is  considered  the 
correct  classification  decision  for  42598;  the  second  is  based  on  a  second  cued  point  (location 
based  on  the  dancing  arrows)  collected  1.37  m  from  the  dynamic  target.  It  was  collected  over  a 
completely  different  dynamic  anomaly  in  an  HD  area  that  was  not  included  in  the  cued  targets 
list  compiled  by  the  QC  Geophysicist,  but  the  result  was  a  0.990  decision  statistic  indicating  a 
3.5-in.  rocket  as  the  predicted  source.  Because  of  the  strong  match  to  what  could  be  a  native 
TOI,  it  was  left  in  the  dig  list.  The  target  ID  was  not  changed  because  the  HD  area  targets  were 
generally  not  given  target  IDs. 


Table  6.2 

Classification  Categories 


Category 

Dig? 

Comment 

0 

Yes 

Inconclusive 

1 

Yes 

High  probability  TOI;  decision  statistic  >  0.925 

2 

Yes 

Possible  TOI  based  any  of: 

1)  Signal  strength  >  20  and  decision  statistic  >  0.825 

2)  Signal  strength  <  20  and  decision  statistic  >  0.750 

3)  Added  based  on  analyst  notes 

3 

No 

Usable  data  not  matching  requirements  for  category  1  or 

2;  does  not  require  re-shot 

4 

No 

Requires  re-shot 
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Table  6.3 

Dig  List  Summary  Statistics 


Statistic 

Project 
(Number  /  %) 

Targets  collected 

1,165 

Targets  classified 

1,137  /  97.51 

Targets  labeled  as  digs 

65  /  5.72 

Inconclusive 

22/  1.92 

Training 

0/02 

Category  1 

12/  l.l2 

Category  2 

31  /  2.72 

T argets  labeled  as  clutter 

1,072  /  94.32 

1  28  targets  identified  as  requiring  re-shots  were  not  re-shot  due 
to  time  and  budget  constraints.  These  were  placed  at  the  end  of 
the  ranked  dig  list  (category  4)  and  were  not  included  in 
statistic  calculations 

2  Percentage  calculated  according  to  number  classified,  not 
number  collected. 
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7.0  PERFORMANCE  ASSESSMENT 


7.1  REPEATABILITY  OF  IVS  MEASUREMENTS 

Seventeen  dynamic  surveys  of  the  IVS  strip  were  performed  during  the  project.  The  results  from 
all  the  first  five  surveys  over  each  IVS  item  were  averaged  to  produce  an  expected  response  for 
each.  The  responses  for  each  item  in  each  survey  were  compared  to  the  expected  response 
plus/minus  25%.  Eleven  response  amplitude  failures  were  noted  during  the  project,  with 
responses  both  higher  and  lower  than  the  acceptable  responses  for  the  IVS  items.  Only  one  data 
location  failure  was  identified,  and  it  was  attributed  to  a  driving  error  rather  than  GPS  failure. 

IVS  response  amplitude  failures  have  been  noted  on  previous  projects,  particularly  with  the 
MetalMapper.  Because  it  is  so  far  in  front  of  the  skid  steer,  slight  changes  in  the  direction  of  the 
skid  steer  result  in  much  more  significant  changes  in  the  location  of  the  sensor.  Because  there 
were  no  other  identifiable  problems  with  the  sensor  during  data  collection  at  PCD,  the  IVS 
response  failures  noted  during  the  project  have  been  attributed  to  the  path  of  the  sensor  over  the 
items  not  being  repeatable  enough  to  produce  a  consistent  response  rather  than  sensor  failure. 

The  industry  has  identified  this  as  a  continuing  problem  and  is  moving  towards  static  functions 
tests  for  QC  of  proper  transmitter  and  receiver  operation  rather  than  dynamic  repeatability  tests. 
Parsons  also  perfonned  function  tests  during  the  project  to  see  if  these  tests  were  a  better  gauge 
of  instrument  functionality  than  the  dynamic  tests.  Function  test  methodology  is  explained  in 
Section  5.4.2.  Function  test  results  were  generally  more  favorable  than  the  dynamic  results, 
although  seven  of  these  tests  also  failed.  However,  all  but  two  of  the  function  test  failures  were 
for  receiver  4,  which  is  the  middle  receiver  in  the  MetalMapper.  This  was  consistently  the 
receiver  farthest  from  the  test  objects.  Because  of  the  receiver  4  failures,  the  locations  of  the  test 
objects  will  need  to  be  refined  for  future  projects,  but  it  seems  that  consistent  function  test 
responses  should  be  achievable  once  the  ideal  locations  are  identified. 

7.2  SPATIAL  COVERAGE 

Spatial  coverage  was  detennined  by  transect  and  by  the  area  covered  within  each  transect 
(collection  was  intentionally  stopped  in  Transect  5  due  to  anomaly  density).  Offsets  were  less 
than  70  cm  for  more  than  99%  of  the  area  covered  in  all  transects  and  less  than  90  cm  for  more 
than  99.7%  of  the  area  covered  in  all  transects. 

7.3  DOWN  TRACK  SPACING 

All  datasets  passed  the  requirements  of  more  than  90%  of  the  point  spacings  less  than  10  cm 
from  each  other  and  98%  of  the  spacings  less  than  15  cm  from  each  other. 

7.4  DETECTION  OF  ALL  TARGETS  OF  INTEREST 

All  seed  items  covered  by  the  dynamic  survey  outside  of  the  analyst  designated  HD  areas  were 
detected  within  40  cm  of  the  measured  emplacement  locations.  Two  seed  items  inside  HD  areas 
on  Transect  9  did  not  have  dynamic  targets  picked  within  40  cm  of  the  measured  emplacement 
locations.  These  were  added  to  the  cued  target  list  by  the  QC  Geophysicist  as  targets  92807  and 
92808  to  test  the  effectiveness  of  cued  data  collection  in  these  areas  despite  the  two  seeds  having 
been  missed  in  the  dynamic  survey. 
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Target  92807,  an  inert  81  -mm  mortar  at  50-cm  depth,  was  definitely  missed  by  the  Blakely  test 
target  picking  algorithm,  which  was  the  only  analysis  performed  inside  the  HD  areas  (i.e.,  the 
analyst  did  not  review  the  targets  picked  within  these  areas).  The  anomaly,  added  target  location 
and  modeled  source  location  for  target  92807  are  shown  in  Figure  7.1.  It  appears  that  nearby 
extremely  high-amplitude  anomalies  caused  the  leveling  routine  to  reduce  the  peak  measured 
over  the  seed  to  the  point  that  it  was  not  automatically  selected  as  a  target. 


Figure  7.1 
Target  92807 


Figure  7.1  shows  the  location  of  added  target  92807  (cross)  and  the  modeled  source  location  from  the  cued  data 

(circle). 
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Figure  7.2 
Target  92808 


Figure  7.2  shows  the  location  of  added  target  92808  (cross)  and  the  modeled  source  locations  (circles)  of 
targets  90019,  90020,  90021,  90022,  and  92808  from  the  cued  data  collected  for  those  targets.  Targets 
90020,  90021,  and  92808  all  had  classifications  suggesting  a  small  ISO  source  with  decision  metrics  higher 

than  0.990. 

It  is  less  clear  if  Target  92808,  a  small  ISO  at  20-cm  depth,  was  actually  missed  in  the  dynamic 
survey.  The  nearest  dynamic  pick  prior  to  the  addition  of  92808  was  90020  (Figure  7.2),  which 
was  53  cm  from  the  measured  emplacement  location  of  the  seed.  Point  92808,  which  was  added 
on  the  anomaly  peak  nearest  the  measured  burial  location,  was  still  15  cm  northeast  of  the 
measured  location.  However,  as  shown  in  Figure  7.2,  the  cued  data  collected  for  targets  90019, 
90020,  90021,  90022,  and  92808  all  suggest  that  the  actual  location  of  the  seed  is  much  closer  to 
the  selected  location  for  target  90020.  Targets  90020,  90021,  and  92808  all  had  classifications 
suggesting  a  small  ISO  source  with  decision  metrics  higher  than  0.990. 

Because  all  seeds  outside  of  HD  areas  had  targets  within  40  cm,  the  project  is  considered  to  have 
passed  the  objective. 
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7.5  CORRECTLY  IDENTIFY  SEED  ITEMS  IN  THE  IVS 

All  of  the  cued  IVS  points  were  identified  as  the  correct  seed  item  in  the  correct  location.  The 
lowest  confidence  metric  recorded  for  the  IVS  points  was  0.96.  The  largest  horizontal  offset  was 
15  cm,  and  the  largest  vertical  offset  was  7  cm. 

7.6  CORRECTLY  POSITION  METALMAPPER  RELATIVE  TO  SOURCE 

Six  targets  required  re-shots  due  to  an  offset  of  greater  than  40  cm  between  the  collection  point 
and  the  nearest  modeled  source.  There  was  insufficient  time  to  collect  re-shots  before  the  end  of 
the  project,  so  these  targets  could  not  be  classified.  All  targets  with  uncollected  re-shots  were 
identified  as  category  4  targets  in  the  ranked  dig  list. 

Forty-seven  targets  that  were  classified  have  collection  point  to  modeled  source  location  offsets 
greater  than  40  cm  but  were  judged  usable  by  the  analyst  based  on  these  possible  factors: 

•  The  target  was  classified  as  TOI  despite  having  a  modeled  location  greater  than  40  cm 
from  the  collection  location. 

•  The  operator  noted  during  collection  that  there  was  no  source  evident  at  the  dynamic 
target  location  (based  on  the  dancing  arrows  display),  so  data  were  collected  directly  on 
top  of  the  dynamic  location  to  satisfy  the  requirements  of  the  collection  to  dynamic  target 
distance  data  quality  objective. 

•  The  target  was  picked  on  a  peak  in  a  larger  anomaly  for  which  at  least  one  other  target 
within  the  anomaly  was  already  classified  as  a  dig. 

•  The  version  of  the  target  selected  for  classification  (multiple  object  solver  version  of  the 
point  selected  rather  than  single  object  solver  version)  was  more  indicative  of  TOI  than  a 
modeled  point  within  40  cm  of  the  collection  location  despite  being  outside  of  the  data 
quality  objective  radius. 

•  The  areal  extent  of  the  anomaly  on  which  the  dynamic  target  was  picked  was  less  than 
0.2  m2. 

7.7  CORRECTLY  POSITION  METALMAPPER  RELATIVE  TO  DYNAMIC  TARGET 

Twenty-two  targets  required  re-shots  due  to  an  offset  of  greater  than  40  cm  between  the 
collection  point  and  the  intended  dynamic  target.  There  was  insufficient  time  to  collect  re-shots 
before  the  end  of  the  project,  so  these  targets  could  not  be  classified.  All  targets  with  uncollected 
re-shots  were  identified  as  category  4  targets  in  the  ranked  dig  list. 

Twenty-three  targets  that  were  classified  have  collection  point  to  dynamic  target  location  offsets 
greater  than  40  cm  but  were  judged  usable  by  the  analyst  based  on  these  possible  factors: 

•  The  areal  extent  of  the  anomaly  on  which  the  dynamic  target  was  picked  was  less  than 
0.2  m2. 

•  A  second  cued  point  was  collected  within  40  cm  of  the  intended  dynamic  target,  but  the 
selected  result  more  closely  matched  a  TOI  in  the  library. 

•  The  operator  could  not  get  any  closer  to  the  dynamic  target  due  to  an  obstacle  (one 
occurrence,  classified  as  a  dig). 
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7.8  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 


Because  an  intrusive  investigation  was  not  part  of  this  project,  the  correct  classification  of  TOI 
could  only  be  judged  according  to  the  seed  items  covered  by  the  cued  survey.  Cued  data  were 
collected  over  10  seed  items,  and  all  were  classified  as  TOI.  Eight  of  the  10  were  classified  as 
category  1  digs,  the  other  two  were  category  2  digs.  The  lowest  decision  statistic  for  any  of  the 
seeds  was  0.880  for  target  92807,  the  81  -mm  mortar  that  required  the  addition  of  a  dynamic 
target  when  it  was  missed  in  the  dynamic  survey. 

7.9  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

The  removal  of  the  10  TOI  from  the  1,137  targets  classified  leaves  a  total  of  1,127  unknown 
sources,  with  55  of  those  classified  as  digs  (22  category  0  [inconclusive]  and  33  category  1  or  2). 
Even  if  all  of  these  end  up  being  false  positives  after  the  intrusive  investigation,  this  represents  a 
95.2%  reduction  in  clutter  digs.  The  receiver  operating  characteristics  curve  for  the  project  is 
shown  in  Figure  7.3.  This  is  a  preliminary  curve  that  considers  all  of  the  unknown  sources 
classified  as  digs  to  be  false  positives. 


Figure  7.3 

Receiver  Operating  Characteristics  Curve 


Receiver  Operating  Characteristics  Curve  -  Pueblo  Chemical  Depot 
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7.10  CORRECTLY  IDENTIFY  TYPE  OF  TOI 

Of  the  10  seeds  covered  by  the  cued  survey,  one  was  identified  incorrectly.  Target  40425  was  a 
small  ISO  at  30  cm  depth  identified  as  a  medium-sized  TOI  based  on  a  best  library  match  to  an 
81 -mm  mortar.  As  shown  in  Figure  7.4,  this  seed  is  in  the  middle  of  an  area  containing 
extremely  high  anomaly  densities.  It  is  expected  that  nearby  sources  contributed  signal  to  the 
measured  data,  making  the  seed  appear  larger  than  it  actually  was.  None  of  the  various  single  or 
multiple  object  realizations  for  this  target  suggested  a  munition  other  than  an  81 -mm  mortar. 

Figure  7.4 
Target  40425 


Figure  7.4  shows  the  location  of  the  dynamic  target  (cross)  and  modeled  source  location  (circle)  for  target  40425. 

7.11  CORRECTLY  PREDICT  ATTRIBUTES  OF  NON-TOI 

Attributes  were  predicted  for  50  non-TOI  targets  selected  by  the  QC  Geophysicist.  No 
determinations  can  be  made  regarding  the  success  of  these  predictions  until  the  intrusive 
investigation  is  completed. 
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7.12  CORRECT  ESTIMATION  OF  TARGET  LOCATION 

The  target  parameters  estimated  in  this  case  were  the  X,  Y,  and  relative  Z  (depth)  coordinates  of 
the  targets.  The  success  criteria  for  this  performance  objective  were  X,  Y  offsets  for  which  1  a 
of  the  dataset  was  less  than  30  cm  and  1  a  of  the  depth  offset  was  less  than  15  cm.  Metrics  were 
only  calculated  for  the  10  seed  items  covered  by  the  cued  survey  given  the  lack  of  an  intrusive 
investigation.  The  a  of  the  horizontal  offsets  was  18  cm,  and  the  a  for  the  vertical  offset  was 
7  cm. 
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8.0  COST  ASSESSMENT 


The  cost  assessment  was  split  into  two  groups:  development  of  the  Demonstration  Plan  and 
Explosive  Site  Plan,  Seeding  and  Surface  Sweep,  and  MetalMapper  costs.  The  MetalMapper 
costs  include  instruments,  surveying,  and  analysis  costs. 

8.1  COST  MODEL 

The  cost  model  for  the  PCD  demonstration  includes  the  total  cost  of  the  project.  The  total  cost 
includes  the  seeding  operation,  MetalMapper  operations,  processing,  and  intrusive  operation. 
Estimates  for  each  operation  are  listed  in  Table  8.1. 
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Table  8.1 

Details  of  Costs  Tracked 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated  Costs 

Demonstration  Plan  and  Explosives  Site  Plan 

Development  and  finalizing  of  the  Demonstration  Plan  and  Explosives  Site  Plan  $58,781 

Seeding  Survey  Costs 

Seed  emplacement/initial 

Costs  for  mobilization,  surface  sweep,  seed 

$30,887 

set  up 

emplacement,  surveying  seeds 

MetalMapper  Survey  Costs 

Survey  costs 

Dynamic  detection  survey  (5.4  acres): 

$44,415 

Field-related  labor  (two  geophysicists  and 
one  UXO  Tech  III),  equipment  setup,  test  pit 
and  IVS  data  collection,  preprocessing, 
initial  target  selections,  non-equipment 
direct  costs  (per  diem,  hotel,  truck  rental 
tractor,  GPS,  MetalMapper,  shipping,  fuel, 
etc.) 

$8, 225/acre 

Survey  costs 

Cued  classification  survey  (1,164  targets): 
Field-related  labor  (two  geophysicists  and 
one  UXO  Tech  III),  equipment  setup,  cued 
data  collection,  preprocessing,  initial  target 
inversion  for  QC  checks,  non-equipment 
direct  costs  (per  diem,  hotel,  truck  rental, 

$34,920 

fuel,  etc.). 

$30/target 

Instrument  rental  costs:  MetalMapper, 

$12,804 

tractor,  GPS,  etc. 

$1 1 /target 

All  processing  and  analysis  performed 

$3,495 

following  the  completion  of  field  activities 

$3/target 

Total  cued  reacquisition  costs 

$44/target 
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9.0  IMPLEMENTATION  ISSUES 


The  two  notable  implementation  issues  noted  during  the  PCD  demonstration  are  discussed 
below. 


•  Neither  the  dynamic  repeatability  tests  collected  at  the  IVS  nor  the  function  tests 
performed  were  as  successful  as  anticipated.  Failed  response  repeatability  tests  have 
been  attributed  to  slight  deviations  in  the  location  of  the  sensor  above  the  seed  items 
due  to  the  difficulty  in  repeating  the  exact  same  line  over  these  objects  given  the 
sensor’s  location  a  significant  distance  ahead  of  the  transport  vehicle.  The  difficulty 
inherent  in  this  process  has  been  noted  on  previous  projects,  with  twice-daily  static 
function  tests  identified  as  a  possible  replacement.  The  use  of  function  tests  was 
attempted  at  PCD,  but  multiple  failures  were  also  noted  with  these  tests.  Function 
test  failures  were  generally  noted  for  receiver  4,  the  receiver  farthest  from  the  two  test 
objects  placed  on  the  sensor  during  testing.  It  is  expected  that  function  tests  will  be 
an  adequate  replacement  for  dynamic  response  tests  once  an  optimal  combination  of 
test  objects  and  locations  for  the  objects  on  the  sensor  are  identified  for  the 
MetalMapper 

•  While  the  efficient  implementation  of  the  dipole  filter  is  a  constantly  evolving 
process,  it  is  clear  that  two  factors  contributed  to  its  poor  performance:  the  extremely 
small  size  of  the  TOI  and  the  extremely  high  anomaly  densities  at  the  site.  The  latter 
complicates  matters  primarily  by  introducing  more  uncertainty  in  the  estimation  of 
the  spatially  varying  background  signal  that  needs  to  be  removed  from  the  data.  Even 
slight  mismatches  in  the  adjacent  survey  line  amplitudes  in  the  less  dense  areas, 
however,  can  act  as  single  dipole  sources  with  very  low  fit  quality.  Since  the  TOI  is 
extremely  small  in  this  case,  the  threshold  is  very  low  and  in  the  process  retains  many 
of  these  artifacts  as  possible  targets.  In  addition,  there  are  many  instances  when  using 
the  3-dipole  model  inversion  doesn’t  appear  to  be  enough  to  identify  all  the  sources  in 
the  data  (even  in  some  of  those  areas  that  were  identified  as  not  too  dense).  This  is 
evident  by  the  fact  that  neither  the  3-dipole  fit  quality  increases  to  an  appreciable 
level,  nor  do  the  resulting  fit  locations  from  neighboring  peaks  converge  on  consistent 
locations. 
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